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Executive  Summary 

In  June,  1993,  Governor  William  Weld  signed  into  law  the  Massachusetts  Education 
Reform  Act  (MERA).  MERA  committed  the  state  to  increasing  aid  to  local  school 
districts,  and  also  launched  a  process  of  comprehensive,  standards-based  reform.  MERA 
was  one  of  the  earlier  such  state  reforms. 

At  the  time  when  MERA  was  passed,  the  Massachusetts  Department  of  Education  had  a 
staff  of  325  FTE's,  down  from  approximately  1,000  in  1980.  MERA  greatly  expanded 
the  state's  role  in  funding  public  education,  specifying  what  students  should  learn,  and 
holding  educators  accountable  for  students'  achievement.  Despite  this  increased  state 
role,  there  have  been  few  additional  resources  devoted  to  building  the  state's 
implementation  capacity.  For  that  reason,  the  Massachusetts  Education  Reform  Review 
Commission  decided  to  sponsor  this  study  of  the  state's  capacity  to  implement  the  far- 
reaching  changes  required  by  Education  Reform. 

Methods.  Our  research  included  analysis  of  documents  related  to  Education  Reform;  in- 
depth  interviews  with  73  current  and  former  state  officials,  association  leaders,  local 
educators,  and  other  stakeholder  representatives;  and  surveys  mailed  to  district-level 
administrators,  building-level  administrators,  and  teachers. 

What  is  "Capacity?"  We  evaluated  the  state's  capacity  to  implement  MERA  by 
assessing  three  key  areas:  the  state's  goals  and  responsibilities  under  the  law,  the 
resources  and  other  organizational  elements  available,  and  stakeholders'  perspectives  on 
goals,  resources,  and  organizational  elements. 

Which  Elements  of  the  State?  Our  research  concerned  the  entities  within  the  state 
government  whose  activities  affect  K-12  education,  with  an  emphasis  on  the 
Massachusetts  Department  of  Education  but  also  including  the  Board  of  Education,  the 
legislature,  the  Governor's  Office,  and  other  agencies  within  the  executive  branch  such  as 
the  Department  of  Revenue. 

Overall  Capacity  Issues 

MERA  greatly  increased  the  state  role  both  in  funding  public  education  and  in  guiding 
the  local  educational  process.  The  state's  role  changed  to  incorporate  setting  of 
curriculum  frameworks  and  holding  schools  accountable  for  student  performance.  In 
order  to  carry  out  its  new  roles,  the  state  (primarily  the  Department  of  Education)  had  to 
use  familiar  policy  instruments  in  new  ways,  and/or  to  a  greater  extent,  than  previously. 
Finally,  because  MERA  was  designed  to  be  a  systemic  reform  of  education,  all  of  the 
various  state  activities  and  policies  needed  to  fit  together  into  a  coherent  whole  based  on 
state  educational  standards.  In  order  to  produce  systemic  reform,  state  authorities  would 
need  both  an  enhanced  ability  to  collect  data  and  organizational  structures  and  functions 
to  facihtate  its  use. 
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Three  sets  of  capacity  issues  cut  across  all  of  the  different  programmatic  elements  of 
MERA. 

Political  Issues.  By  "political  issues,"  we  mean  issues  related  to  the  climate  of  opinion 
and  assumptions  within  which  state  goals  for  MERA  implementation  have  been  set  and 
local  implementation  has  taken  place.  The  important  political  issues  for  MERA 
implementation  have  been  (1)  the  extent  to  which  influential  actors  agree  with  the 
conception  of  MERA  as  a  "bargain"  between  state  and  local  authorities,  (2)  the 
appropriate  extent  of  the  state  role,  (3)  the  direction  of  leadership  at  the  state  level,  and 
(4)  local  educators'  response  to  the  new  state  roles. 

•  When  MERA  was  passed,  many  state  leaders  assumed  that  the  state's  role  would 
primarily  be  to  set  standards  and  hold  schools  accountable  for  meeting  them,  to  be 
fairly  hands-off  regarding  how  schools  met  the  standards  (the  "tight-loose" 
assumption). 

•  Increasingly,  this  "tight-loose"  approach  is  seen  as  insufficient.  Many  respondents 
argue  that  the  state  has  a  key  technical  assistance  responsibility  to  help  schools  with 
the  processes  of  implementing  standards-based  reform. 

•  Disagreement  over  DOE's  appropriate  role(s)  has  hampered  DOE's  ability  to  receive 
and  strategically  deploy  the  resources  it  needs. 

•  The  change  in  Board  of  Education  leadership  in  1996  heightened  controversy  around 
Education  Reform  and  reduced  educators'  willingness  to  cooperate  with  the  state. 
This  adversely  affected  DOE's  capacity  to  leverage  changes  at  the  local  level. 

•  Local  educators  see  many  of  the  functions  included  in  MERA  as  appropriately  shared 
between  state  and  local  authorities..  In  the  area  of  assessment  there  may  be  resistance 
to  an  increased  state  role,  and  there  is  more  interest  in  DOE  brokering  rather  than 
providing  professional  development. 

Organizational  Issues.  Even  in  the  purest  version  of  the  "tight-loose"  conception  of 
MERA,  the  state  still  needs  to  have  the  capacity  to  produce  and  update  curriculum 
frameworks,  assess  student  performance,  certify  educators,  and  hold  schools  accountable 
for  student  performance.  As  mentioned  above,  there  is  significant  interest  in  the  state 
going  beyond  this  minimum  role  to  support  schools  and  districts  with  technical 
assistance  in  implementing  standards-based  curriculum  and  instruction. 

Many  different  entities  within  state  government  have  had  some  role  in  MERA 
implementation.  The  Department  of  Education  occupies  a  complicated  place,  in  that  it 
has  had  to  answer  to  the  Board  of  Education,  the  Govemor  (since  it  is  an  executive 
branch  agency),  a  now-defiinct  Secretary  of  Education,  and  the  legislature  (which  funds 
it),  while  at  the  same  time  needing  to  remain  responsive  to  the  Commonwealth's  many 
schools  and  school  districts. 
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• 


This  organizational  complexity  has  contributed  to  competing  and  fluctuating  policy 
agendas  for  DOE  and  local  educators. 

This  organizational  complexity  has  also  prevented  DOE  from  making  the  case  to  the 
legislature  for  additional  administrative  resources. 

Within  DOE,  communication  among  offices  and  coordination  of  efforts  have  been 
problematic.  Staff  see  this  as  due  to  resource  constraints,  a  resulting  workload  that 
leads  to  a  program-specific  focus  by  staff,  and  a  need  for  more  organization-wide 
management.  Combined  federal  and  state  responsibilities  complicate  this. 

Some  educators  in  the  field  perceive  DOE  as  difficult  to  communicate  with  and  to  get 
information  from. 


Financial  and  Human  Resource  Issues.  Limited  funding  and  staff  have  posed  major 
challenges  for  the  implementation  of  MERA. 

Outside  of  the  Department  of  Education,  relatively  few  state  staff  work  on  K-12 
educational  issues.  For  that  reason,  our  analysis  of  financial  and  human  resources  for 
MERA  implementation  focuses  primarily  on  DOE.  MERA  added  significantly  to  DOE's 
responsibilities,  but  did  not  take  any  functions  away  from  the  Department.  These 
increased  responsibilities  were  to  be  carried  out  by  a  staff  that  had  been  greatly  reduced 
over  the  previous  decade.  DOE  staff  increases  were  slow  in  coming:  a  net  increase  of 
only  13  FTE  between  1993  and  1998,  followed  by  an  increase  of  67  FTE  between  1998 
and  2001. 


• 


• 


About  45%  of  DOE  staff  are  in  positions  paid  with  federal  funds,  which  in  most  cases 
restricts  the  agency's  ability  to  use  them  for  implementing  state  programs.  In 
addition,  the  overhead  funds  that  come  with  these  federal  grant  funds  go  not  to  DOE 
but  to  the  state's  General  Fund. 

Over  95%  of  new  state  spending  under  MERA  has  been  "passed  through"  DOE  to 
local  districts,  with  very  little  left  over  for  building  state-level  administrative  capacity 
or  evaluating  what  works  and  does  not  work. 

The  legislature  has  passed  numerous  other  initiatives  in  addition  to  MERA. 
Administrative  and  evaluation  funds  are  often  lacking  in  these  initiatives. 


•  Both  inside  and  outside  DOE,  there  is  widespread  belief  that  the  agency  does  not 
have  sufficient  staff  to  implement  MERA  and  its  other  responsibilities. 

•  Because  the  state  pay  scale  is  lower  than  what  experienced  educators  can  earn  in 
schools  and  districts,  it  is  difficult  to  recruit  people  with  extensive  experience,  in 
schools  or  elsewhere,  for  jobs  at  DOE.  DOE  has  responded  to  staffing  needs  through 
consultants  and  sabbatical  teachers. 
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•  Use  of  consultants  and  sabbatical  teachers  has  ahnost  certainly  not  closed  the  gap 
between  the  level  of  DOE's  regular  staff  and  the  extent  of  its  responsibilities. 

•  Accountability,  data  collection,  research  and  evaluation,  and  school  building 
assistance  are  areas  in  which  the  need  for  more  state-level  staff  is  particularly  acute. 

Issues  Specific  to  Components  of  Education  Reform 

At  the  beginning  of  our  research  process,  we  analyzed  the  Education  Reform  Act  and 
identified  ten  categories  of  state-level  activity.  We  then  examined  the  capacity  issues 
within  each  area. 

Developing  and  Implementing  Standards  of  Learning.  This  area  has  included  the 
production  of  the  Common  Core  of  Learning,  Curriculum  Frameworks  and  other 
standards  such  as  vocational  standards,  as  well  as  supporting  local  districts' 
implementation  of  standards  through  alignment  of  curriculum  and  instruction. 


• 


Standards-based  reform  involves  both  development  and  implementation  components. 
Although  this  process  has  at  times  been  politically  contentious,  it  is  administratively 
easier  than  helping  districts  implement  the  standards. 

•  The  initial  Common  Core  of  Learning  process  demonstrates  the  power  of  a 
collaborative  process  to  inspire  cooperation  in  the  field  (which  in  turn  expands  the 
capacity  of  the  state  to  foster  standards-based  reform).  Subsequent  estrangement  of 
the  field  limits  this  important  avenue  for  expanding  state  capacity. 

•  Because  of  the  small  numbers  of  staff  involved,  DOE's  efforts  to  support 
implementation  have  not  been  extensive.  Local  educators  would  like  more  help. 

Developing  and  Implementing  a  System  of  Student  Assessment.  The  model  of 
systemic  reform  enacted  by  MERA  requires  an  assessment  of  what  students  have  learned, 
to  serve  as  an  indicator  of  whether  reform  is  succeeding. 


• 


Like  many  other  state  education  authorities,  DOE  has  used  a  test  contractor  to  build 
its  capacity  in  this  area. 

Even  with  test  production  contracted  out,  extensive  responsibilities  remain  in  DOE, 
for  which  current  staff  levels  are  barely  sufficient. 

Local  educators  express  dissatisfaction  with  the  state's  ability  to  help  them  interpret 
MCAS  results. 
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Developing  an  Accountability  System  for  School  and  District  Performance. 

Accountability  is  one  of  the  most  important  components  of  MERA,  and  it  is  one  of  the 
areas  in  which  implementation  to  date  has  been  most  hindered  by  political, 
organizational,  and  resoiirce  issues. 

•  Because  of  limited  resources,  the  state  accountability  system  emphasizes  oversight  of 
a  few  schools  that  appear  to  have  the  most  egregious  problems,  rather  than  the 
Education  Reform  principle  of  continuous  improvement  for  all  schools  in  the 
Commonwealth. 

•  Competition  and  conflict  between  branches  and  offices  of  the  state  government  has 
slowed  implementation  of  an  accountability  system  for  schools  and  districts. 

•  The  Department  of  Education  lacks  resources  needed  for  broad-based  technical 
assistance  to  schools  and  districts  on  school  improvement. 

Supporting  Local  Education  Governance  and  Management.  MERA  included 
significant  changes  in  the  way  schools  and  districts  are  run.  All  schools  are  to  have 
School  Councils.  School  Committees'  power  over  personnel  issues  was  reduced,  with 
superintendents  and  principals  given  more  authority.   DOE's  role  in  these  local  changes 
was  to  provide  support  for  districts  while  also  ensuring  that  the  new  laws  were  followed. 

•  Districts  have  less  intemal  capacity  for  planning  and  governance  than  MERA 
assumed. 

•  School  Councils  and  other  govemance  and  management  reforms  have  been  unevenly 
implemented. 

•  School  committees  would  benefit  from  professional  development  on  their  role  imder 
MERA. 

•  The  state's  own  limited  capacity  prevents  it  from  supporting  local  capacity-building. 

Making  Coherent  State-Level  Policy.  MERA  required  not  only  that  the  structures  and 
processes  of  the  K-12  public  education  system  change,  but  also  that  they  change  in  a 
systematic  and  coherent  way  consistent  with  standards-based  reform.  In  order  for  state 
authorities  to  play  their  role  in  coordinating  local  efforts,  they  must  themselves  act  in  a 
coordinated  way  to  guide  and  monitor  education  activities. 

•  State-level  policymaking  has  been  a  contentious  area,  with  competing  agendas, 
fluctuating  goals,  and  a  gradual  estrangement  of  the  field. 

•  The  policymaking  structure  has  been  in  flux,  with  the  Board  of  Education  changing 
from  17  to  9  members,  the  Secretariat  of  Education  being  abolished,  and  the  Board's 
Advisory  Committees  having  an  unclear  role 
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•  The  state  could  improve  interagency  coordination  and  communication  to  further 
reform's  effectiveness  and  coherence. 

Collecting  and  Using  Information  to  Improve  the  Performance  of  the  Educational 
System.  Coordination  of  policy  making,  accountability,  and  communication  requires  that 
the  state  collect  appropriate  information  and  use  it  in  ways  that  contribute  to  system 
improvement. 

•  DOE  has  focused  significant  resources  on  technology  and  information  systems,  but 
the  basic  functions  of  knowing  what  schools  students  attend,  what  courses  they  take, 
and  how  they  perform  on  assessments  have  not  yet  been  implemented. 

•  MERA  placed  significant  responsibility  in  this  area  with  the  Secretary  of  Education, 
which  was  abolished,  and  with  DOE's  then-division  of  Evaluation,  Planning,  and 
Research,  which  has  been  reorganized  and  receives  few  resources  for  this  function. 

•  Other  entities  within  state  government,  such  as  the  Joint  Committee  on  Education, 
Arts,  and  Humanities  and  the  Education  Reform  Review  Commission,  do  not  have 
sufficient  capacity  for  data  collection. 

Enhancing  Educator  Quality  and  Education  as  a  Profession.  The  approach  taken 
thus  far  to  professional  development  in  MERA  exemplifies  the  "tight-loose"  assumption 
about  the  state's  role  in  reform.  Local  districts  received  state  funds  for  professional 
development  and  were  expected  to  use  it  in  ways  that  supported  reform. 

•  In  practice,  many  districts  have  not  known  how  to  use  the  fimds  effectively,  and  the 
state  has  had  insufficient  resources  to  use  in  guiding  them. 

•  DOE  has  had  relatively  Httle  professional  development  resources  of  its  own  because 
most  state  funds  for  professional  development  go  to  local  districts.  DOE  has  begun 
building  capacity  in  its  Educator  Quality  cluster;  several  interviewees  spoke  highly 
about  its  initial  progress. 

•  The  field  has  seen  accountability  issues,  including  MCAS  and  teacher  testing,  taking 
precedence  over  professional  development  support. 

Ensuring  Readiness  to  Learn  Through  Early  Childhood  Education  Programs. 

Between  1996  and  1999,  spending  on  the  early  childhood  education  component  of 
MERA  increased  by  247%. 

•  A  report  by  the  State  Auditor  concluded  that  monitoring  and  evaluation  capacity  did 
not  keep  pace  with  the  spending  increase. 

•  Survey  and  interview  respondents  expressed  the  opinion  that  early  childhood 
education  programs  still  need  evaluation,  improvement  and  expansion. 
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Implementing  Choice  and  Charter  Schools.  MERA  expanded  interdistrict  choice  and 
authorized  the  state  to  approve  charter  schools. 

•  Although  the  charter  school  office  staff  is  quite  small,  it  has  produced  an 
accountability  system  that  some  people  believe  could  provide  guidance  for  a  more 
general  accountability  system. 

Funding  Public  Education.  When  MERA  passed  in  1993,  many  observers  expressed 
doubt  that  the  state  would  make  good  on  its  promise  to  spend  $1.3  billion  in  new  state 
funds  on  education  over  seven  years. 

•  The  funding  promise  was  fulfilled,  and  all  districts  have  now  been  brought  up  to  the 
foundation  level  of  education  spending. 

•  The  state  has  recognized  that  the  definition  of  the  foimdation  level  must  be  revised  to 
keep  up  with  the  challenge  of  standards-based  reform. 

Recommendations 

•  Increase  the  funds  available  for  DOE  operations,  and  use  the  increased  funds  to 
improve  the  agency's  capacity  in  key  areas.  DOE  has  particularly  strong  resource 
needs  in  the  areas  of  data  collection  and  management,  research  and  evaluation,  and 
assessment.  Other  important  areas  of  need  include  staff  for  the  school  building 
assistance  program  and  funds  to  be  used  as  state  matches  for  federal  grants. 


• 


Ensure  that  legislatively  mandated  programs  include  adequate  funds  for  DOE  to 
administer  and  evaluate  them.  In  general,  whenever  state  government  creates  a 
new  grant  program  or  imposes  a  new  responsibility  or  reporting  requirement  on  the 
DOE  or  local  school  districts,  it  should  specify  a  source  of  fimding  for  administrative 
support,  including  resources  for  facilitation,  oversight,  and  evaluation  of  the 
program — ^perhaps  5%  for  administration  and  3%  for  evaluation. 

•  Allow  overhead  from  DOE's  Federal  grants  to  go  to  DOE  rather  than  the 
General  Fund.  Other  potential  sources  of  funds  include  the  revenues  firom  educator 
certification  fees,  which  also  currently  go  to  the  General  Fund  rather  than  Department 
administrative  costs. 

•  Salaries  for  DOE  staff  should  be  increased,  in  order  to  attract  greater  numbers 
of  experienced  educators  to  work  in  the  agency.  DOE  is  not  able  to  match  what 
experienced  superintendents,  principals,  and  teachers  can  eam  in  school  districts. 
This  is  a  difficult  issue  to  resolve,  given  that  the  salary  ranges  for  DOE  are  generally 
the  same  as  for  all  state  agencies.  DOE  has  creatively  used  contracted  employees, 
sabbatical  teachers,  and  other  mechanisms  to  address  this  shortcoming,  but  the 
capacity  costs  of  these  practices  cannot  be  underestimated.  Given  the  ambitious 
combination  of  support  and  regulatory  roles  required  of  DOE  by  MERA,  an  inability 
to  hire  sufficient  numbers  of  experienced  educators  is  a  significant  limitation. 
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•  Increase  involvement  of  and  communication  with  educators  in  the  field  in 
implementation  of  Education  Reform.  DOE  has  already  begun  to  take  some 
initiative  in  this  area.  Overall,  the  relationship  between  state  authorities  and  the  field 
needs  to  become  less  adversarial.  From  the  governor,  the  legislature,  the  Board,  the 
Department,  and  the  various  education  associations,  there  needs  to  be  some  sort  of 
common  attempt  to  say  that  Education  Reform  is  a  worthy  goal,  that  we  have  much  to 
celebrate,  that  we  are  going  to  get  there,  and  that  we  will  move  forward  together. 

•  Where  appropriate,  use  resources  and  organizations  outside  of  DOE  to  expand 
state  capacity.  In  some  areas  where  DOE  lacks  capacity,  such  as  building  assistance, 
auditing,  professional  development,  and  other  types  of  technical  assistance  there  is 
likely  to  be  relevant  expertise  elsewhere,  either  through  other  state  agencies  or 
outside  contractors.  Using  networks  of  outside  providers  would  still  require  DOE  to 
have  adequate  financial,  human,  and  technological  resources  to  develop  and  manage 
the  networks  and  to  monitor  quality  of  service.  It  is  also  important  to  recognize  the 
limitations  of  contracting  out  as  a  means  of  policy  implementation. 

•  Resolve  uncertainty  over  responsibility  for  accountability,  monitoring,  and 
oversight  In  addition  to  establishing  a  clear  sense  of  which  roles  will  be  played  by 
the  Govemor's  Office  and  which  by  the  DOE,  it  is  also  necessary  to  determine  how 
both  entities'  work  fits  with  that  of  the  State  Auditor  and  the  Education  Reform 
Review  Commission.  In  the  area  of  district  accountability,  lessons  may  be  leamed  by 
comparing  the  different  accountability  methods  used  to  evaluate  charter  schools  and 
regular  public  schools. 

•  Maintain  and  enhance  DOE's  capacity  to  play  a  regulatory  role.  Emphasis  on 
DOE  as  a  supporter  or  facilitator  of  improved  teaching  and  learning  must  be  seen  as 
complementing  rather  than  supplanting  its  role  as  the  enforcer  of  federal  and  state 
regulations.  Although  networks  outside  the  agency  can  assist  in  the  support  and 
facilitating  functions  and  in  service  delivery,  legal  compliance  remains  DOE's  legal 
responsibility  as  a  state  agency. 

•  Expand  the  use  of  sampling  in  data  collection  and  program  evaluation.  Closely 
monitoring  the  activities  of  371  districts  and  1,874  schools  is  a  daunting  task. 
However,  there  may  be  numerous  areas  in  which  a  sampling  strategy  can  deliver 
representative  data  at  lower  capacity  cost  than  collecting  data  from  all  districts  and/or 
schools.  Adequate  data  collection  and  analysis  through  a  sampling  strategy  would 
probably  still  require  additional  DOE  capacity,  but  less  than  would  be  required  by  a 
census  approach. 
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Improve  coordination  and  communication  within  DOE.  Many  interviewees 
(inside  and  outside  DOE)  talked  about  their  impression  that  the  work  of  various  DOE 
clusters  and  staff  members  is  not  well  coordinated,  both  because  there  isn't  a  chief 
operating  officer  or  staff  member  with  similar  knowledge  of  and  authority  over  which 
cluster  is  doing  which  tasks  and  because  communication  among  clusters  is  largely  ad 
hoc.  The  Commissioner's  recent  reorganization  addresses  one  of  these  issues  by 
adding  a  chief  operating  officer  to  the  organizational  chart.  This  reorganization,  and 
subsequent  ones,  may  also  make  cross-cluster  communication  and  collaborations 
more  routine. 

Consider  regional  or  at  least  western  and  southeastern  Massachusetts  field 
offices.  At  present,  all  of  DOE,  including  the  Program  Quality  Assurance  staff,  whose 
areas  of  responsibility  are  geographically  defined,  are  based  in  Maiden.  There  may 
be  significant  benefits  to  be  gained,  in  terms  of  building  better  connections  between 
DOE  and  local  educators,  from  a  more  regional  approach.  Regionally  based  staff 
need  not  become  disconnected  from  Maiden,  but  could  use  information  technology 
for  internal  and  external  communication  and  coordination. 

Conduct  additional  research.  Areas  of  needed  research  include  the  followiag: 
How  are  districts  spending  their  MERA  fimds? 

What  would  a  data  management  system  optimally  look  like,  based  on  the  data-use 
needs  of  both  the  Department  and  local  educators? 

What  are  the  best  practices  in  professional  development  now,  and  how  could  a 
provider  network  be  organized  to  bring  these  practices  into  classrooms  across  the 
state? 

-  Evaluation  of  early  childhood  programs  and  spending — ^what's  the  best  use  of 
state  funds? 

-  Is  there  a  fiscal  crisis  coming,  due  to  the  large  number  of  current  school  building 
projects,  that  will  drain  instructional  resources  in  the  future? 


When  MERA  was  passed  in  1993,  its  proponents  had  a  vision  of  a  future  in  which  all 
Massachusetts  children  would  have  equal  educational  opportunities  and  would  be  able  to 
meet  world-class  academic  standards.  The  path  to  that  future  has  proved  long  and 
challenging.  Several  of  the  state  policy  makers  we  met  with  described  Education  Reform 
implementation  as  having  recently  reached  "the  hard  part,"  or  as  passing  through 
"adolescence."  They  remained  hopeful,  however,  that  the  challenges  of  Education 
Reform  can  be  surmounted.  We  hope  that  the  analysis  and  recommendations  presented 
in  this  paper  will  be  used  constructively  to  support  the  Commonwealth  in  meeting  these 
challenges. 
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I.       Introduction 

In  June,  1993,  Governor  William  Weld  signed  into  law  the  Massachusetts  Education  Reform  Act 
(MERA).  MERA  committed  the  state  to  increasing  aid  to  local  school  districts,  and  also 
launched  a  process  of  comprehensive,  standards-based  reform.  Since  the  early  1990s,  many 
other  states  have  begun  similar  reform  initiatives,  but  MERA  was  one  of  the  first. 

At  the  time  when  MERA  was  passed,  the  Massachusetts  Department  of  Education  had  a  staff  of 
325  FTE's,  or  only  33%  as  large  as  in  1980.  Public  education  had  been  seen  as  a  local  matter, 
with  only  a  small  state  role.  MERA  greatly  expanded  the  state's  role  in  funding  public 
education,  specifying  what  students  should  learn,  and  holding  educators  accountable  for 
students'  achievement.  Despite  this  increased  state  role,  there  have  been  few  additional 
resources  devoted  to  building  the  state's  implementation  capacity.  For  that  reason,  the 
Massachusetts  Education  Reform  Review  Commission  decided  to  sponsor  this  study  of  the 
state's  capacity  to  implement  the  far-reaching  changes  required  by  Education  Reform. 

The  remainder  of  this  introductory  section  outlines  the  methods  we  used,  our  definition  of 
"capacity,"  which  parts  of  the  state's  education  system  we  emphasized  in  our  research,  and 
which  MERA-related  functions  we  studied.  In  Part  II,  we  identify  ten  key  policy  components  of 
MERA  and  identify  capacity-related  implementation  challenges  common  to  all  MERA  policy 
areas.  In  Part  HI,  we  discuss  capacity  issues  specific  to  each  often  primary  tasks  given  to  the 
state  under  MERA.  In  Part  IV,  we  present  recommendations  for  enhancing  the  state's  capacity 
to  implement  MERA. 

Methodology 

The  first  component  of  our  research,  document  analysis,  entailed  collecting  as  many  Education 
Reform-related  reports  and  documents  as  we  could  obtain,  from  both  state  government  and 
private-sector  sources.  A  complete  list  of  these  documents  follows  the  text  of  this  report.  In 
analyzing  these  documents,  we  paid  particular  attention  to  what  we  could  learn  about  use  of 
resources,  lines  of  authority,  and  state  and  local  understandings  of  MERA  as  it  was  being 
enacted  and  implemented. 

Second,  we  carried  out  interviews  with  73  current  and  former  state  officials,  association  leaders, 
local  educators,  and  other  stakeholder  representatives,  in  March,  April,  and  May  2001.  All 
interviews  but  one  were  conducted  in  person.  We  used  a  common  set  of  interview  questions  and 
topics  to  guide  all  interviews,  but  did  not  necessarily  discuss  the  same  issues  in  the  same  order 
with  all  interviewees.  All  interviewees  were  promised  anonymity.  Unless  otherwise  noted, 
information  included  in  this  report  comes  from  the  interviews.  A  list  of  interviewees  is  included 
in  Appendix  5. 

Third,  our  research  included  three  surveys  of  local  educators:  superintendents,  principals,  and 
teachers.  Survey  questions  addressed  the  fit  between  state  and  local  goals  under  Education 
Reform,  the  relative  importance  of  different  Education  Reform  components,  data  sharing  and 
communication  with  the  Department  of  Education,  and  the  appropriate  balance  between  state  and 
local  roles  in  education  policy. 
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The  Massachusetts  Association  of  School  Superintendents,  the  Massachusetts  Elementary  School 
Principals  Association,  the  Massachusetts  Association  of  Secondary  School  Principals,  the 
Massachusetts  Teachers  Association,  and  the  Massachusetts  Federation  of  Teachers  all  supported 
the  survey  component  of  this  project  by  urging  their  members  to  participate  and  by  sharing 
mailing  lists.  For  more  information  on  our  sampling  strategy  for  the  surveys,  see  the 
introductory  section  of  Appendix  2. 

Superintendents  had  a  response  rate  of  41%,  with.  113  of  175  surveys  returned.  565  surveys  were 
sent  to  principals  and  1 67  of  them  v^ere  returned  in  time  for  data  analysis,  resulting  in  a  return 
rate  of  30%.  Teachers'  response  rate  was  much  lower.  We  sent  616  surveys  out  and  received  98 
back,  for  a  response  rate  of  16%.  The  fiill  survey  data  analysis  is  included  in  Appendix  2  of  this 
report. 

Finally,  we  conducted  an  informal  study  of  the  staffing  and  organization  of  other  programs  with 
similar  objectives  to  parts  of  Education  Reform,  as  well  as  of  comparisons  of  state  education 
agencies.  Our  comparative  analysis  of  programs  in  other  jurisdictions  was  not  a  systematic 
search  for  best  practices  nationwide.  Rather,  it  consisted  of  seeking  out  information  on  a  few 
programs,  selected  because  of  our  familiarity  with  them  or  because  interviewees  in 
Massachusetts  mentioned  them  as  important  models.  Discussion  of  these  other  programs 
appears  in  Appendix  1 . 

What  is  "Capacity?" 

In  our  research,  we  emphasized  not  the  actual  outcomes  of  Education  Reform,  but  rather  the 
resources,  processes,  organizational  arrangements,  and  personnel  that  state  authorities  have  used 
in  pursuit  of  those  outcomes.  Our  analysis  is  informed  by  the  literature  on  "learning 
organizations,"  which  emphasizes  that  organizational  learning  requires  not  just  acquisition  of 
information,  but  also  building  of  a  capacity  to  gather  and  organize  data  as  well  as  to  act 
effectively  (Marquardt,  1996).  We  understand  organizational  capacity  as  a  measure  of  an 
organization's  or  system's  ability  to  use  its  resources  and  processes  to  achieve  its  goals  (Kaplan 
&  Norton,  1992;  Shaw,  1992).  In  the  specific  context  of  MERA,  we  evaluated  whether  the 
resources  and  organizational  characteristics  of  Massachusetts  state  government,  primarily  but  not 
only  the  Department  of  Education,  have  been  sufficient  to  allow  the  state  to  implement  the 
policies  and  carry  out  the  roles  implied  or  demanded  by  the  law. 

We  organized  our  study  around  three  key  factors  in  assessing  capacity: 


•  Goals  and  Responsibihties:  In  order  to  draw  useful  conclusions  about  the  adequacy  of 
organizational  elements,  it  is  necessary  to  understand  the  ends  to  which  the  organizational 
elements  are  means.  The  same  organization  or  system  might  have  adequate  capacity  to  carry  out 
some  of  its  tasks  and  inadequate  capacity  for  others. 

•  Organizational  Elements:  These  include  human,  financial,  and  technological  resources,  as  well 
as  the  structures  and  processes  that  support  their  use.  One  comparative  study  of  state  education 
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agencies  implementing  complex  reform  identified  people,  training,  and  infrastructure  as  the 
necessary  components  of  capacity  (Lusi,  1997).  The  important  organizational  elements  are  not 
only  within  individual  agencies,  but  also  in  the  relationships  among  state  agencies  and  between 
agencies  and  the  field  (Massell,  1998a). 

The  organizational  elements  we  examined  included  staff  and  funding  levels  of  the  Department  of 
Education,  division  of  labor  among  state  agencies,  communication  between  state  and  local 
authorities,  and  the  collection  and  use  of  data. 

•  Perspectives:  An  important  means  of  understanding  capacity  is  analysis  of  stakeholders'  and 
participants'  perspectives  on  whether  goals  have  been  met  and  organizational  elements  are 
sufficient.  It  is  also  important  to  learn  whether  actors  at  different  levels  of  the  system  have  the 
same  or  different  perspectives  on  policies,  as  sharp  differences  of  perspective  will  impede 
implementation. 

Who  Is  "The  State?" 

In  the  United  States,  local  and  state-level  authorities  have  historically  shared  the  responsibility 
for  public  education.  The  broadest  possible  definition  of  "the  state  education  system"  would 
include  all  of  these  state  and  local  authorities,  including  public  higher  education  institutions.  We 
did  not  use  such  a  broad  definition  in  this  research.  Instead,  we  focused  on  the  capacity  and 
activities  of  entities  within  the  state  government  whose  activities  affect  K-12  education,  with  an 
emphasis  on  the  Massachusetts  Department  of  Education  but  also  including  the  Board  of 
Education,  the  legislature,  the  Governor's  Office,  and  other  agencies  within  the  executive  branch 
such  as  the  Department  of  Revenue. 

Capacity  to  Do  What? 

As  in  the  other  New  England  states,  control  over  and  responsibility  for  funding  K-12  education 
in  Massachusetts  has  historically  rested  with  local  authorities.  MERA  greatly  increased  the  state 
role  both  in  funding  public  education  and  in  guiding  the  local  educational  process,  including 
curriculum,  student  assessment,  and  evaluation  of  school  performance. 

Some  of  the  most  significant  changes  to  the  state  role  were  a  commitment  to  use  state  funds  to 
bring  all  districts  in  the  state  up  to  a  foundation  level  of  per-pupil  spending  by  2000, 
recertification  of  educators  every  five  years  rather  than  lifetime  certification,  publication  of  more 
imiform  information  about  schools  and  districts,  fimding  and  managing  a  charter  schools 
program,  establishment  of  statewide  educational  goals  and  standards,  increased  instructional 
time,  a  requirement  that  students  pass  an  examination  based  on  the  state  standards  before 
graduating  from  high  school,  and  statewide  Certificates  of  Mastery  and  of  Occupational 
Proficiency.  State  authorities  were  also  involved  in  implementation  of  changes  in  the  roles  of 
actors  at  the  local  level,  such  as  superintendents,  principals.  School  Committees,  and  the  newly 
created  School  Councils. 

Drawing  on  the  Center  for  Education  Policy's  previous  work  in  producing  a  conceptual  map  of 
MERA,  we  divided  the  many  tasks  the  Act  required  of  the  state  government  into  ten  key 
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categories:  1)  developing  and  implementing  standards  of  learning;  2)  developing  and 
administering  a  system  oi  student  assessment;  3)  developing  an  accountability  system  for  school 
and  district  performance;  4)  supporting  and  changing  local  governance  and  management  of 
public  education;  5)  making  effective  state  level  policy  decisions;  6)  collecting  and  using 
information  to  inform  policy  decisions  and  improve  the  performance  of  the  state's  educational 
system  ;  7)  enhancing  educator  quality  and  education  as  a  profession  through  new  processes  for 
certification  and  professional  development;  8)  ensuring  readiness  to  learn  through  early 
childhood  and  literacy  programs;  9)  encouraging  school  choice  and  charter  schools;  and  10) 
dX\0Q,2i\mg  funding  in  a  way  that  is  adequate  and  equitable,  both  within  and  across  districts. 

In  Part  in  of  the  report,  we  will  return  in  detail  to  these  ten  categories  of  tasks.  In  Part  11,  we 
will  examine  several  capacity-related  challenges  that  have  been  common  to  all  ten  areas. 
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il.      Overall  Issues  in  State  Capacity 

In  carrying  out  their  new  duties  under  MERA,  the  Department  of  Education  (DOE)  and  other 
state  agencies  have  employed  a  wide  range  of  poUcy  instruments,  including  the  following: 

State  Curriculum  Frameworks 

State  Financial  Aid  to  School  Districts 

The  Massachusetts  Comprehensive  Assessment  System  (MCAS) 

Audits 

Assistance  to  Underperforming  Schools 

Certification  and  Recertification  Requirements 

Professional  Development 

Regulations 

Guidelines 

Competitive  Grants 

Contracts 

Reporting  Requirements  for  Schools  and  Districts 

Although  the  state's  MERA  responsibilities  were  new,  many  of  these  policy  instruments  were 
not.  For  example,  prior  to  MERA  the  state  already  had  a  standardized  assessment,  the  MEAP. 
The  Department  of  Education  had  administered  competitive  grants  and  contracts  before,  and  had 
worked  with  professional  development  providers. 

MERA  wrought  three  major  kinds  of  changes  in  the  state's  use  of  these  pohcy  instruments. 
First,  the  state's  role  changed  to  incorporate  setting  of  curriculum  frameworks  and  holding 
schools  accountable  for  student  performance.  Second,  under  MERA  the  state  (primarily  the 
Department  of  Education)  had  to  use  familiar  policy  instruments  in  new  ways,  and/or  to  a  greater 
extent,  than  previously.  Where  the  MEAP,  for  example,  had  produced  school-level  scores,  the 
MCAS  produces  scores  for  individual  students  that  reflect  their  mastery  of  the  state  curriculum 
frameworks,  which  are  also  used  to  identify  under-performing  schools.  Or,  in  the  field  of 
accountability,  DOE  had  for  years  had  a  Program  Quality  Assurance  unit  monitoring  district 
compliance  with  federal  and  state  regulations.  Under  MERA,  state  authorities  would  examine 
not  only  legal  compliance,  but  also  the  extent  to  which  school  personnel  were  able  to  work 
effectively  with  students. 

Finally,  because  MERA  was  designed  to  be  a  systemic  reform  of  education,  all  of  the  various 
state  activities  and  policies  needed  to  fit  together  into  a  coherent  whole  based  on  state 
educational  standards.  Data,  in  particular  student  assessment  results,  would  drive  systemic 
improvement.  In  order  to  produce  systemic  reform,  state  authorities  would  need  both  an 
enhanced  ability  to  collect  data  and  organizational  structures  and  fimctions  to  facilitate  its  use. 

Three  major  sets  of  issues — political,  resource-related,  and  organizational — have  complicated 
the  state's  efforts  to  expand  its  role  in  education  policy  in  a  systematic  way. 
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Political  Issues 

By  "political  issues,"  we  mean  issues  related  to  the  climate  of  opiaion  and  assumptions  within 
which  state  goals  for  MERA  implementation  have  been  set  and  local  implementation  has  taken 
place. 

According  to  state-level  leaders  we  interviewed,  a  confluence  of  three  factors  led  to  the 
legislature's  passage  of  MERA  in  1993.  First,  local  educational  authorities'  budgets  had  been 
cut  dramatically  during  the  1980s  as  a  result  of  Proposition  2V2  limits  on  local  taxation.  The 
Massachusetts  Association  of  School  Superintendents  (MASS)  wanted  the  state  to  increase  its 
aid  to  districts  as  way  of  making  up  for  the  revenue  shortfalls.  Second,  inequities  in  the  financial 
resources  available  to  property-rich  and  property-poor  districts  had  inspired  a  lawsuit,  McDuffy 
V.  Robertson,  which  many  policy  makers  expected  the  state  to  lose.  They  wanted  to  enact 
financial  reform  before  the  court  acted,  rather  than  wait  for  a  court-ordered  solution.  Finally, 
business  leaders  had  become  highly  critical  of  the  public  schools'  performance  and  organization. 
The  Massachusetts  Business  Alliance  for  Education  (MBAE)  issued  a  report  in  1991  entitled 
"Every  Child  a  Wirmer,"  whose  recommendations  the  legislature  used  as  a  guide  in  drafting 
MERA.  MASS  joined  with  MBAE  in  supporting  the  bill,  but  some  of  the  other  education 
associations  opposed  it  because  of  the  changes  it  made  to  teachers'  and  principals'  certification 
and  job  security. 

The  important  political  issues  for  MERA  implementation  have  been  the  extent  to  which 
influential  actors  agree  with  the  conception  of  MERA  as  a  "bargain"  between  state  and  local 
authorities,  the  extent  to  which  DOE's  role  should  be  a  regulatory  one,  the  direction  of 
leadership  at  the  state  level,  and  local  educators'  response  to  the  new  state  roles. 

MERA  as  a  "Bargain"  of  Resources  for  Accountability 

Some  have  characterized  the  model  of  state-local  relations  underlying  MERA  as  a  "bargain,"  in 
which  the  state  gave  districts  more  money  and  a  reduced  regulatory  burden  in  exchange  for 
accountability  for  student  performance  on  state  assessments.  Another  way  of  describing  the 
model  is  "tight-loose":  the  state  is  "tight"  in  its  identification  of  goals,  but  "loose"  in  regulating 
what  schools  and  districts  do  to  achieve  those  goals.  The  tight-loose  model  assumes  that  local 
districts  have  the  capacity  to  make  and  implement  effective  decisions  in  response  to 
accountabiUty  pressure  from  the  state. 

The  alternative  to  the  "tight-loose"  state-local  relationship  is  one  in  which  the  state  takes  a  much 
more  hands-on  role  in  local  districts.  In  this  vision,  the  state  would  not  stop  at  monitoring 
student  performance  on  standardized  tests.  It  would  also  regulate  how  districts  spent  MERA 
funds,  punish  them  for  deviating  firom  regulations,  and  specify  how  districts  should  address  the 
curriculum  frameworks. 

This  gap  in  perceptions  matters  because  the  two  different  understandings  of  the  state's  role  in 
MERA  lead  to  different  conceptions  of  what  the  state  ought  to  be  doing  and  how  much  capacity 
it  needs  to  carry  out  its  responsibilities.  If  the  role  of  the  state  is  only  to  set  standards  and  to  get 
out  of  the  districts'  way  while  they  work  to  meet  them,  there  is  less  of  a  case  to  be  made  for 
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expanded  state  capacity  than  if  the  state  is  seen  as  needing  to  be  closely  involved  with  local 
implementation  of  curriculum  frameworks  and  other  elements  of  Education  Reform. 

Few  in  Massachusetts  hold  to  the  strictest  version  of  either  the  tight-loose  or  interventionist  state 
models,  and  in  a  sense  the  dichotomy  is  a  false  one.  Even  the  "tight-loose"  model  requires 
sufficient  state  capacity  to  manage  a  statewide  assessment  and  apply  sanctions  to 
underperforming  schools.  However,  the  point  remains  that  different  understandings  of  how 
large  the  state  role  was  supposed  to  be  under  MERA  lead  to  different  senses  of  how  much 
implementation  capacity  the  state  needs.  Based  on  our  interviews,  what  looks  to  some  people 
like  a  "tight-loose"  partnership  between  state  and  local  education  authorities  feels  to  some  local 
educators  like  the  state  has  imposed  mandates  on  them  without  providing  any  assistance  with 
how  to  respond.  In  a  sense,  this  local  perception  parallels  many  DOE  employees'  belief  that 
MERA  gave  them  more  responsibility  without  a  commensurate  increase  in  resources. 

Over  the  course  of  MERA  implementation,  some  policy  makers  have  come  to  question  the 
assumptions  behind  the  "tight-loose"  understanding  of  Education  Reform.  Specifically,  they 
now  beheve  that  not  all  local  school  districts  have  the  capacity  to  decide  how  to  bring  all 
students'  achievement  up  to  high  standards.  As  one  interviewee  put  it: 

The  schools  are  overwhelmed  with  all  they  have  to  do.  Don 't  assume  they  know  best  how  to 
do  Ed  Reform  in  their  schools.  There 's  very  little  planning,  or  analysis  of  what 's  working. 
Schools  need  help — time  to  plan,  resources  to  evaluate,  change  what  they  're  doing.  [There 
are]  schools  that  are  underperforming,  have  had  audits  done — I  don 't  know  what  happens 
after  that.  There  aren  't  the  kind  of  support  systems  that  need  to  happen  for  underperforming 
schools — more  resources,  experts  to  help  schools  work  through  problems — schools  are  trying 
to  do  figure  it  out  on  their  own  and  running  in  a  lot  of  directions. 

Some  districts  and  local  educators  have  also  expressed  a  need  for  more  direction  firom  the  state. 
In  a  way,  this  is  beginning  to  happen,  as  the  legislature  has  made  some  education  funding  more 
categorical  in  nature  than  had  previously  been  the  case.  To  the  extent  that  the  state  takes  a  larger 
role  in  directing  local  activities,  more  state  capacity  will  be  needed  to  ensure  that  this  direction  is 
of  high  quality. 

Regulation,  Support,  or  Partnership? 

According  to  many  of  our  interviewees,  the  most  important  question  to  be  answered  about  the 
state's  role  in  MERA  implementation  is  whether  the  Department  of  Education  should  emphasize 
regulation  or  support  and  technical  assistance  in  its  relations  with  local  districts.  As  one  person 
put  it,  DOE  is  both  a  "cop"  and  a  "social  worker."  In  addition  to  asking  which  fimctions  DOE 
should  emphasize,  it  is  also  necessary  to  ask  whether,  and  with  what  other  entities,  DOE  should 
share  both  kinds  of  function.  If  DOE  is  not  the  only  "cop"  or  "social  worker"  in  the  education 
system,  then  how  should  those  roles  be  shared,  and  what  resources  are  necessary?  This  issue  is 
not  unique  to  Massachusetts.  Research  in  other  states  suggests  that  the  role  of  regulation  in 
standards-based  school  reform  has  firequently  been  problematic  (Elmore,  Abelmann,  and 
Fuhrman,  1996,  pp.  90-91). 


State  Capacity  to  Implement  the  Massachusetts  Education  Reform  Act  of  1993 


In  the  1980s  and  early  1990s,  many  educators  in  the  field  believed  that  DOE  was  excessively 
regulatory.  Through  the  1990s,  key  figures  in  the  leadership  of  DOE  tried  to  move  away  firom 
that  role.  However,  some  of  our  interviewees  still  think  DOE  does  too  much  monitoring  and 
regulating,  as  opposed  to  supporting  local  change  efforts  in  more  positive  ways.  Particularly  in 
the  last  few  years,  the  push  toward  more  accountability  has  shifted  DOE  out  of  a  support  role  in 
the  eyes  of  many  local  educators.  Even  within  the  Department,  interviewees  mentioned  what 
they  called  a  "natural  progression"  from  reform  elements  in  which  DOE  acted  as  a  partner  with 
the  field,  such  as  designing  the  Common  Core  of  Learning  and  the  Curriculum  Frameworks,  to 
functions  such  as  assessment  and  accountability  that  put  DOE  into  a  more  regulatory  role. 
Determining  the  architecture  of  reform,  while  not  an  easy  task,  creates  less  political  conflict  than 
does  assessing  its  strength. 

Others  believe  that  DOE  does  not  do  enough  monitoring  and  regulating  in  specific  policy  areas 
that  contribute  to  major  priorities  of  Education  Reform.  For  example,  the  Department  does  not 
have  comprehensive  information  on  how  districts  have  spent  the  extra  fimds  they  received 
through  MERA.  In  particular,  many  districts  have  failed  to  spend  the  state  aid  they  have 
received  for  professional  development.  Coming  at  a  time  when  educators  face  the  unprecedented 
challenge  of  high  standards  for  all  students,  the  shortfall  threatens  the  success  of  the  reform  (see 
Massachusetts  Teachers'  Association,  2000;  also  Education  Management  and  Accountability 
Board,  2000).  School  Councils,  another  key  component  of  Education  Reform,  have  been 
implemented  only  "on  paper"  in  many  districts,  and  have  not  taken  on  the  strong  role  envisioned 
for  them  in  MERA.  Where  a  given  individual  stands  on  the  issue  of  DOE' s  appropriate 
regulatory  role  influences  how  much  capacity  she  or  he  believes  the  state  needs  in  order  to 
implement  MERA.  However,  even  in  the  most  minimal  conception  of  the  state's  role,  it  still 
must  have  capacity  to  enforce  regulations  and  monitor  pohcy  implementation. 

Lack  of  staffing  (which  will  be  discussed  in  more  detail  below)  has  been  a  major  reason  for 
DOE's  inabihty  to  regulate  and  monitor.  The  lack  of  staff  increases  for  DOE  following  passage 
of  MERA,  which  many  people  agree  produced  a  major  capacity  problem  (see  below),  was  in  part 
due  to  a  particular  understanding  of  the  agency's  appropriate  regulatory  role.  In  the  mid- 1990s, 
key  leaders  in  the  executive  branch  (including  DOE  leadership)  believed  that  a  larger  agency 
would  be  a  more  regulatory  one,  and  that  a  more  regulatory  agency  would  be  a  bad  thing.  Thus, 
they  did  not  seek  additional  resources  for  DOE.  However,  constraining  DOE's  regulatory 
capacity  by  limiting  its  staffing  also  constrains  its  ability  to  support  local  MERA 
implementation. 

Some  view  MERA  implementation  as  taking  place  in  a  "distributed  system,"  with  responsibility 
and  authority  shared  broadly  between  state  and  local  authorities.  Under  the  leadership  of  Martin 
Kaplan,  who  was  Chair  when  MERA  passed,  the  Massachusetts  Board  of  Education  was 
committed  to  participation  and  involvement  of  the  field.  Several  of  our  interviewees  described 
the  Kaplan  board's  process  for  creation  of  the  Common  Core  of  Learning  as  a  model  of  building 
shared  capacity.  Over  a  nine-month  period,  the  40-member  Commission  on  the  Common  Core 
of  Learning  involved  approximately  50,000  people  in  hearings  and  meetings  around  the  state. 

Literature  on  policy  implementation  in  general,  and  education  policy  implementation  in 
particular,  emphasizes  the  need  for  participation  and  mutual  adjustment  between  policy  makers 
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and  policy  implementers.  Administrators  and  teachers  have  a  great  deal  of  autonomy;  in  such  a 
"loosely  coupled"  system  (Weick  1976),  it  is  difficult  if  not  impossible  to  force  them  to  comply 
with  top-down  policies.  Because  coercion  is  ineffective  in  educational  institutions,  several 
recent  studies  of  systemic  reform  in  education  have  highlighted  the  need  to  include  teachers' 
disposition  towards  change  in  implementation  strategies  (Goertz,  Floden,  and  O'Day  1996)  and 
to  treat  local  administrators  in  ways  that  predispose  them  to  act  as  allies  rather  than  impediments 
to  reform  (Massell,  Kirst,  and  Hoppe  1997).  As  Lusi  (1997)  points  out,  persuasion  and  joint 
learning  by  state  and  local  officials  should  be  considered  as  policy  instruments  for  complex, 
state-led  reform  (170). 

Although  the  Common  Core  was  not  mandated  by  MERA,  it  was  intended  as  a  starting  point  for 
deliberation  on  the  required  Curriculum  Frameworks.  When  Governor  Weld  replaced  Kaplan 
with  John  Silber  as  Chair  of  the  Board  of  Education,  the  process  became  much  less  collaborative. 
Silber  had  been  appointed,  in  part,  because  Weld  believed  Education  Reform  implementation 
was  not  moving  fast  enough.  Broad  participation  was  seen  as  having  produced  much  of  the 
delay.  Under  Silber,  state  authorities  rejected  the  original  frameworks  for  being  insufficiently 
rigorous,  too  laden  with  educational  jargon,  and  not  specific  enough  to  measure  with  a 
standardized  assessment.  Ironically,  the  controversies  over  these  changes  introduced  still  more 
delay  to  MERA  implementation.  Since  they  were  first  completed,  the  Curriculum  Frameworks 
have  been  revised  several  times  through  a  much  less  participatory  process,  and  DOE  no  longer 
distributes  the  Common  Core  of  Learning  to  districts. 

According  to  some  of  our  interviewees,  these  changes  in  the  Frameworks  deprived  state 
authorities  of  locally  based  capacity  to  implement  (in  other  words,  deprived  the  state  of  the 
power  of  a  distributed  system).  The  initial  participatory  process  had  built  a  cohort  of  educators 
who  were  excited  about  the  Frameworks  and  looking  forward  to  implementing  them  in  their 
classrooms.  After  the  abrupt  changes,  these  educators  felt  betrayed  and  viewed  the  Frameworks 
as  an  unwelcome  state  mandate.  As  a  result  of  the  shift  in  policy  and  reduction  of  input  from  the 
field,  many  educators  have  come  to  believe  that  the  state's  goals  for  MERA  have  been  unstable 
and  subject  to  capricious  change. 

Leadership 

The  theme  of  shifts  in  leadership  emerged  numerous  times  in  our  interviews.  Many  interviewees 
(especially  those  outside  state  government)  see  Education  Reform  as  having  changed  course  in 
1996.  From  1993  to  1996,  Board  Chair  Martin  Kaplan  promoted  collaboration  between  state 
education  authorities  and  the  field.  In  1996,  John  Silber  replaced  Martin  Kaplan  as  Chair  of  the 
Board  of  Education  and  the  Board's  membership  was  reduced  from  17  to  9.  The  new  appointees 
included  several  people  known  to  be  critics  of  the  public  schools  or  affiliated  with  the  Pioneer 
Institute,  a  Boston  research  organization  that  had  advocated  school  privatization.  Relations 
between  the  state  and  the  field  became  much  more  conflictual. 

Silber's  term  as  Board  Chair  also  contrasts  with  the  situation  since  1999,  with  David  Driscoll  as 
Commissioner  of  Education  and  James  Peyser  as  Board  Chair.  Since  this  second  major  change 
in  leadership,  the  Board  and  the  Department  have  begun  rebuilding  connections  between  the 
state  and  the  field  and  undoing  some  of  the  damage  done  to  local  educators'  faith  in  the  state  by 
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Silber's  public  statements  and  policy  decisions.  However,  our  research  indicates  that  there  is 
still  much  work  to  be  done  in  the  area  of  rebuilding  relations  with  the  field. 

The  Board  of  Education  is  the  main  source  of  change  in  state  leadership  of  Education  Reform. 
Outside  the  Board,  there  has  been  continuity  in  most  of  the  other  leadership  positions.  Most  of 
the  legislative  leaders  responsible  for  passing  MERA  have  moved  on,  but  the  legislature  remains 
solidly  in  Democratic  control.  There  have  been  three  different  govemors  or  acting  govemors 
since  1993,  but  relative  continuity  in  administrations.  Current  Governor  Jane  Swift  was 
Lieutenant  Governor  under  her  predecessor,  Paul  Cellucci,  who  in  turn  had  previously  been 
Lieutenant  Govemor  imder  William  Weld,  who  signed  the  Education  Reform  Act  as  Govemor. 

Even  had  there  been  total  stability  in  leadership,  educators  and  the  public  might  still  perceive  a 
shift  in  the  priorities  of  Education  Reform  in  the  late  1 990s  when  the  implementation  process 
reached  what  several  of  our  interviewees  called  the  "hard  parts:"  assessment  and  accountability. 
Although  there  have  certainly  been  important  changes  in  leadership  of  the  Board  of  Education, 
the  importance  of  stability  in  other  areas — especially  maintenance  of  the  legislature's 
commitment  to  increased  state  funding  for  public  education — should  not  be  underestimated. 

At  least  since  the  begiiming  of  the  last  century,  education  reformers  have  wanted  to  keep  poHtics 
out  of  the  schools.  In  an  era  of  comprehensive  reform  led  by  state  elected  and  appointed 
officials,  which  challenges  long-held  assumptions  about  local  control  of  education,  political 
influence  on  policy  implementation  is  inevitable.  Disagreement  over  the  state-local  division  of 
power,  the  extent  to  which  DOE's  role  should  be  regulatory,  and  the  overall  direction  and  goals 
of  Education  Reform  have  affected  both  the  state's  implementation  capacity  and  the  difficulty  of 
the  tasks  confronting  the  state. 


Local  Educators'  Views  of  State  and  Local  Responsibilities 

Despite  controversy  over  Education  Reform,  local  educators  in  our  survey  generally  accepted  the 
idea  of  sharing  responsibility  for  MERA-related  tasks  with  state  authorities. 
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Table  1  -  The  Education  Reform  Act  of  1993  assigned  many  new  education-related  tasks  to  the  state 
government.  For  each  category  of  tasks  in  the  following  table,  please  indicate  whether  you  believe  these 
tasks  should  be  primarily  a  state  responsibility,  primarily  a  local  responsibility,  or  a  shared 
responsibility.  * 


Task 

Primarily 

State 

Responsibility 

Shared 
Responsibility 

Primarily 

Local 

Responsibility 

Developing  and  implementing  curriculum  standards 

12.1 

76.1 

11.5 

Developing  and  administering  a  student  assessment  system 

10.7 

72.8 

16.3 

Developing  an  accountability  system  for  school  and  district 
performance 

14.6 

72.5 

12.6 

Supporting  local  governance  and  management  of  education 
reform 

14.2 

63.2 

19.7 

Developing  and  administering  processes  for  certifying, 
evaluating,  and  dismissing  school  personnel 

14.8 

52.6 

31.8 

Providing  and/or  guiding  professional  development 

2.3 

58.6 

38.5 

Allocating  funding  in  a  way  that  is  adequate  and  equitable 

42.6 

47.2 

9.7 

Facilitating  inter-district  choice 

19.7 

27.9 

37.1 

Supervising  charter  schools 

58.5 

22.2 

8.5 

Ensuring  readiness  to  learn  through  early  childhood  and  literacy 
programs 

11.0 

70.9 

15.8 

Making  state-level  policy  and  planning  decisions  in 
coordination  with  key  actors 

31.6 

62.6 

1.4 

Collecting  and  using  information  to  improve  the  performance  of 
the  state  educational  system 

30.4 

65.6 

2.8 

*  "Don't  know"  responses  are  omitted. 

Of  the  twelve  tasks  presented  in  the  survey,  at  least  a  plurality  of  local  educators  said  that  ten 
should  be  shared  state-local  responsibilities.  A  majority  viewed  supervision  of  charter  schools  as 
a  state  responsibility,  and  a  plurality  said  that  facilitating  inter-district  choice  should  be  primarily 
a  local  responsibility.  Although  a  plurality  believed  that  responsibility  for  equitable  and 
adequate  allocation  of  funding  should  be  shared,  almost  as  large  a  proportion  of  respondents 
identified  it  as  primarily  a  state  responsibility. 

Nearly  three-quarters  of  our  respondents  said  that  curriculum  standards  and  school  accountability 
should  be  shared  state  and  local  responsibilities.  Of  respondents  who  did  not  prefer  a  shared 
responsibility  in  these  areas,  slightly  more  said  they  ought  to  be  primarily  state  responsibilities 
than  said  they  ought  to  be  primarily  local  responsibilities. 

As  in  curriculum  standards  and  school  accountability,  about  three-quarters  of  the  respondents 
expressed  the  belief  that  responsibility  for  student  assessment  should  be  shared  between  state  and 
local  authorities.  However,  people  who  saw  assessment  as  primarily  either  a  state  or  local 
responsibility  were  likelier  to  see  it  as  primarily  local  rather  than  primarily  state.  This  is  the 
opposite  of  the  responses  for  curriculum  standards  and  school  accountability. 

These  survey  fmdings  are  important  because  they  suggest  that  although  local  educators  see  many 
of  the  functions  included  in  MERA  as  appropriately  shared  between  state  and  local  authorities,  in 
a  few  key  areas  such  as  professional  development  and  assessment  there  may  be  resistance  to  an 
increased  state  role.  Such  resistance  makes  the  "tight-loose"  model  of  MERA  implementation 
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less  likely  to  work  because  of  its  reliance  on  local  initiative  to  respond  to  the  challenge  of 
standards-based  education. 

Organizational  Issues 

MERA  assigned  several  crucial  new  roles  to  the  Massachusetts  state  government,  in  particular 
the  Department  of  Education.  Even  in  the  purest  version  of  the  "tight-loose"  conception  of 
MERA,  the  state  still  needs  to  have  the  capacity  to  produce  and  update  curriculum  frameworks, 
assess  student  performance,  certify  educators,  and  hold  schools  accountable  for  student 
performance.  Going  beyond  this  minimum  role,  the  state  arguably  must  also  support  schools  and 
districts  with  technical  assistance  and  professional  development. 

Implementation  of  MERA  has  involved  many  organizational  entities  within  the  executive  and 
legislative  branches  of  state  government.  The  Department  of  Education  is  the  agency  with 
responsibility  for  the  majority  of  MERA  implementation  activities,  in  addition  to  its  other  work 
managing  a  variety  of  separate  initiatives  of  the  federal  government  and  the  state  legislature, 
such  as  special  education,  bilingual  education,  school  nutrition,  and  a  host  of  others. 

The  Department  receives  policy  directives  regarding  MERA  activities  from  three  different  state 
sources — ^the  Board  of  Education,  the  Govemor,  the  Governor's  education  staff,  and  the 
legislature — as  well  as  responding  to  media  stories  and  the  demands  of  its  local  educator 
constituencies.  Although  the  existence  of  the  Board  of  Education  makes  it  appear  that  DOE  is  a 
quasi-independent  agency,  it  is  in  fact  part  of  the  executive  branch,  which  the  Govemor  heads. 
DOE's  leaders  must  publicly  support  policy  decisions  made  by  the  Govemor  and  cannot  make 
independent  appeals  to  the  legislature  for  increased  fiinding  or  other  resources. 

The  Board  of  Education  has  played  a  very  strong  role  in  MERA  decisionmaking,  with  hands-on 
involvement  in  areas  including  curriculum  frameworks,  MCAS,  underperforming  schools, 
educator  licensure,  and  approval  of  charter  schools.  The  Board  is  a  lay  board  of  9  individuals 
(reduced  from  17  in  1996)  appointed  by  the  govemor  for  5-year  terms,  with  one  dedicated  staff 
person,  based  in  the  Department.  This  person's  primary  responsibilities  are  preparation  of 
meeting  materials  and  keeping  the  Board  agenda. 

The  Governor's  Office  has  an  education  advisor  and  a  deputy  education  advisor  (and  until 
recently,  the  Lieutenant  Govemor),  who  have  conducted  policy  research,  given  policy  directives 
to  the  Department,  and  worked  to  influence  the  Board's  activity.  As  of  this  year,  the  Govemor's 
Office  is  also  the  new  locus  of  the  state's  school  and  district  accountability  activities.  In  January, 
2001,  former  Govemor  Cellucci  signed  a  law  creating  an  independent  Office  of  Educational 
Quality  and  Accountability  with  a  $3.9  million  budget  and  a  5-member  oversight  board.  The 
oversight  board  is  now  headed  by  James  Peyser,  who  is  also  Chair  of  the  Board  of  Education  and 
the  education  advisor  to  Govemor  Jane  Swift  (Miller,  2001).  It  is  possible  that  the  shift  of 
accountability  into  the  Govemor's  Office  and  the  triple  role  being  played  by  Peyser  signal  a  shift 
in  power  towards  the  Govemor's  Office  and  away  from  tiie  Board  of  Education. 

Proponents  of  the  independent  accountability  office  believe  that  it  is  necessary  to  separate 
technical  assistance  and  regulatory  functions  in  this  area.  They  also  believe  that  more  objective 
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evaluation  of  schools'  performance  will  come  from  outside  the  Department  of  Education. 
Opponents  of  shifting  accountability  out  of  DOE  argue  that  the  Department  had  developed  a 
workable  accountability  model  and  was  building  internal  capacity  to  use  it,  and  that  the  shift 
merely  added  a  layer  of  bureaucracy  and  set  the  scene  for  intragovemmental  competition. 

The  Department  of  Revenue's  Division  of  Local  Services  has  also  played  a  role  in  MERA 
accountability.  In  1997,  then-govemor  Weld  issued  Executive  Order  393,  which  created  the 
Education  Management  Accountability  Board  and  charged  the  DOR's  Division  of  Local  Services 
to  provide  a  team  of  auditors  to  that  effort.  An  Audit  Bureau  was  set  up,  which  now  numbers  14 
FTE  and  has  issued  fiscal  and  operational  audit  reports  on  23  local  districts  (6  in  partnership  with 
DOE  accountability  staff)  under  the  Education  Management  Accoimtability  Board  (EMAB). 
Because  of  the  creation  of  the  new  accountability  council  in  the  Governor's  Office,  EMAB  is  no 
longer  in  operation. 

The  Massachusetts  legislature  plays  a  policy  and  finance  role  in  the  implementation  of  MERA. 
The  legislative  committee  with  jurisdiction  on  these  issues  is  the  Joint  Committee  on  Education, 
Arts  &  Humanities,  with  a  staff  of  3.  Its  impact  on  implementation  has  included  continued 
appropriation  of  MERA's  school  finance  formula  and  its  moving  of  fimding  for  MERA's 
accountability  fimction  from  the  Department  of  Education  to  the  governor's  office.  Within  the 
past  year,  the  legislature  also  has  taken  the  initiative  in  appointing  a  new  Foundation  Budget 
Review  Commission.  This  Commission,  required  by  MERA,  has  the  responsibility  for  continual 
review  of  the  adequacy  of  the  state's  foundation  budget.  Governor  Weld  appointed  the  first 
Commission,  within  the  executive  branch,  but  allowed  its  members'  terms  to  expire  without 
appointing  new  members.  The  new  version  of  the  Foundation  Budget  Review  Commission  is 
within  the  legislative  branch,  co-chaired  by  the  House  and  Senate  chairs  of  the  Committee  on 
Education,  Arts,  and  Humanities. 

The  Massachusetts  Education  Reform  Review  Commission  (MERRC)  was  established  by 
MERA  and  charged  with  monitoring  the  extent  to  which  MERA  implementation  has  occurred 
and  has  "brought  about  education  reform  in  the  Commonwealth."  It  has  a  diverse  membership, 
including  the  chairs  of  the  Joint  Committee  on  Education,  Arts  &  Humanities,  representatives  of 
various  education  groups,  higher  education,  parents,  and  others.  MERRC  has  a  mandate  to 
conduct  research  on  MERA's  implementation,  but  it  did  not  receive  fimding  for  this  purpose  until 
FY  1997  and  its  fimding  levels  remain  very  low:  $300,000  total  budget  for  FY  2002. 

A  final  state  player  in  the  implementation  of  MERA  is  the  Board  of  Higher  Education,  which 
sets  policies  for  state  colleges  and  universities  affecting  teacher  education,  professional 
development,  school  partnerships,  and  entrance  requirements  for  incoming  college  students.  The 
Commissioner  of  Higher  Education  is  an  ex-officio  member  of  the  state's  K-12  Board  of 
Education. 

Organization  of  the  Department  of  Education 

The  Department  of  Education  itself  has  been  the  most  important  organization  in  MERA 
implementation.  Its  roles  have  included  producing  the  curriculum  frameworks,  managing  the 
contractors  who  produced  the  MCAS  and  the  Massachusetts  Educator  Certification  Tests, 
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administering  new  educator  certification  programs  and  regulations,  providing  technical 
assistance  to  schools  and  districts,  and,  until  this  year,  preparing  to  hold  schools  and  districts 
accountable  for  students'  MCAS  performance. 

Over  the  past  thirty  years,  the  Department  of  Education's  structure  has  changed  in  two  key  ways. 
First,  the  size  of  its  staff  has  decreased  dramatically  (see  below  for  details),  but  at  the  same  time 
it  has  had  an  increasing  number  of  staff  at  or  above  the  Associate  Commissioner  level.  Second, 
prior  to  1991  it  had  Regional  Centers,  but  now  all  of  its  staff  are  based  in  Maiden. 

DOE  has  been  reorganized  numerous  times,  and  a  fiill  analysis  of  the  changes  is  beyond  the 
scope  of  this  study.  However,  we  obtained  organizational  charts  from  1970,  1988,  1991,  1994, 
1998,  and  2000  for  comparison  with  the  agency's  structure  in  2001  (prior  to  the  most  recent 
reorganization  announced  in  May).  Several  of  the  areas  in  which  DOE's  capacity  has  recently 
been  questioned  (in  particular,  research,  evaluation,  and  data  collection — described  below)  are 
ones  that  seem  to  have  lost  priority  on  the  organizational  charts  either  before  or  shortly  after 
passage  of  MERA. 

In  1970,  the  Department's  senior  staff  consisted  of  the  Commissioner,  one  Deputy 
Commissioner,  two  Associate  Commissioners  (for  Administration  and  Persormel,  and  for 
Curriculum  and  Instruction),  and  four  Assistant  Commissioners  (for  Research  and  Development, 
School  Facilities  and  Related  Services,  Occupational  Education,  and  State  and  Federal 
Assistance).  Each  of  the  Associate  and  Assistant  Commissioners  headed  a  Division.  The 
Regional  Centers  were  part  of  the  Bureau  of  Elementary  and  Secondary  Education,  within  the 
Division  of  Curriculum  and  Instruction.  By  1988,  there  were  still  six  Divisions,  but  with  a 
mostly  different  lineup  of  responsibilities:  Educational  Personnel,  Finance  and  Administration, 
Occupational  Education,  School  Programs,  School  Services,  and  Special  Education.  There  were 
also  six  offices  reporting  to  the  Commissioner:  the  Adult  Education  Bureau,  the  Community 
Education  Office,  the  Educational  Equity  Office,  the  General  Counsel's  Office,  the  Planning, 
Research,  and  Evaluation  Office,  and  the  External  Affairs  Office.  The  Regional  Centers  reported 
directly  to  the  Deputy  Commissioner.  The  Curriculum  Service  Bureau  was  part  of  the  School 
Programs  Division. 

Budget  cuts  hit  DOE  hard  in  the  late  1980s  and  early  1990s.  The  Regional  Centers  were 
eliminated  over  a  three-year  period,  with  the  last  one  closed  down  in  1991.  On  the  1991  DOE 
organizational  chart,  two  divisions  (Finance  and  Administration  and  School  Services)  carry  the 
notation  "due  to  reductions  in  appropriations  units  have  been  inoperable  and  their  function 
assumed  by  other  offices."  The  two  inoperable  divisions  included  some  of  the  fimctions  that 
would  later  be  problematic  in  MERA  implementation,  such  as  data  collection  and  school  audit 
services.  One  interviewee  described  the  cuts  as  follows: 

Those  regional  centers,  when  you  see  what  they  could  do  with  Ed  Reform,  that  was  where  the 
practitioners  went.   They  went  to  the  regional  branch  when  they  had  questions  about 
implementation,  whatever,  professional  development,  they  went  to  the  specialists  in  the 
regions.  And  those  regions  are  gone.  For  example,  we  have  a  problem  in  Pittsfield,  the 
nearest  help  is  Boston.,  So  that  has  to  have  an  impact  in  terms  of  being  able  to  provide 
technical  assistance  in  professional  development,  in  accountability,  in  realigning  the 
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curriculum.  .  .  .  And  then  you  get  this  huge  law,  and  the  administration  would  never  ask  for 
additional  staff. 

According  to  the  DOE's  1993  MERA  Implementation  Plan,  the  Department  was  to  be 
reorganized  along  the  lines  of  its  new  duties.  A  schematic  diagram  in  the  Plan  (p.  21)  shows  four 
overlapping  circles  labeled  "Commissioner  and  Deputy  Commissioner,"  "Education 
Improvement,"  "Evaluation,  Planning,  and  Research,"  and  "Administration  and  Program 
Support."  However,  in  the  1994  Table  of  Organization,  the  Evaluation,  Planning,  and  Research 
area  is  no  longer  on  the  same  level  as  Education  Improvement  and  Administration  and  Program 
Support,  each  of  which  had  an  Associate  Commissioner.  Instead,  the  third  Associate 
Commissioner  is  in  charge  of  Finance  and  Accountability.  Under  that  person,  there  is  an 
Accountability  and  Evaluation  Services  area,  but  the  planning  and  research  functions  are  not 
clearly  placed  on  the  chart.  On  the  1998  "Revised  Organizational  Chart,"  there  are  only  two 
Associate  Commissioners,  for  Program  Accountability  and  Curriculum  Standards.  Most  of  the 
internal  support  functions  such  as  human  resources  and  financial  management  were  in  the  charge 
of  three  other  senior  staff,  the  Chief  Financial  Officer,  the  Chief  of  Staff,  and  the  Chief 
Technology  Officer. 

The  trend  toward  fewer  Associate  Commissioners  reversed  between  1998  and  2000.  The  June 
13,  2000  Organizational  Chart  shows  two  Deputy  Conmiissioners  (for  Administration  and  Policy 
and  for  Academic  Affairs),  a  Senior  Associate  Commissioner  who  is  also  the  General  Counsel,  a 
Chief  Technology  Officer,  a  Chief  Financial  Officer,  and  eight  other  staff  at  the  Associate 
Commissioner  level,  in  charge  of  Accountability,  Education  Program  Services,  Educator  Quality, 
School  Readiness,  Student  Assessment  Programs,  School  to  Career,  Charter  Schools,  and 
Mathematics,  Science,  and  Technology/Engineering.  This  basic  structure  is  the  one  that  was  in 
place  in  May,  2001  when  Commissioner  Driscoll  announced  the  newest  reorganization. 

MERA  assumes  that  state  education  authorities  will  provide  comprehensive,  systematic  guidance 
and  monitoring  of  local  reform  efforts.  However,  some  characteristics  of  DOE's  organization 
(apart  from  its  low  staffing  levels)  seem  to  impede  its  ability  to  play  such  a  role.  In  our 
interviews,  several  people  noted  that  DOE  lacks  an  "operational  ombudsman"  with  a  broad  sense 
of  how  all  the  ongoing  projects  relate  to  each  other  and  final  authority  over  which  offices  should 
perform  which  tasks.  This  used  to  be  the  Deputy  Commissioner's  role,  but  was  lost  when  then- 
Deputy  Commissioner  Driscoll  moved  into  the  top  position  and  the  two  new  Deputy  positions 
were  created.  The  newly  announced  organizational  structure  responds  to  this  problem  by  adding 
a  Chief  Operating  Officer. 

The  other  DOE  organizational  issue  that  frequently  came  up  in  interviews  was  the  lack  of  an 
institutionalized  way  of  sharing  information  across  related  projects.  One  senior  staff  member 
commented  that  there  are  no  incentives  for  sharing  information  among  colleagues,  so  many 
people  allow  it  to  become  a  low  priority.  Educators  in  the  field  complained  of  often  getting 
different  answers  to  questions  from  different  parts  of  DOE,  also  indicating  a  lack  of 
commimication.  The  changes  to  DOE's  structure  over  the  years  have  made  it  a  "flatter" 
organization,  at  least  on  paper,  but  it  may  also  have  simply  increased  the  degree  of 
compartmentalization  among  staff  who  report  to  different  Associate  Commissioners.  As  one 
interviewee  put  it, 
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There 's  a  great  deal  of  Balkanization  within  the  Department.   There  is  poor  communication 
among  people  who  are  working  on  different  things  that  could  be  complementary.  Somehow, 
and  it's  sort  of  a  recurring  problem,  somehow  there  needs  to  be  leadership  at  the 
Department  and  structures  that  create  a  much  more  open  and  collaborative  way  of  looking  at 
issues  so  that,  for  example,  if  we  have  an  issue  about  how  are  kids  going  to  pass  math,  we 
have  to  really  be  able  to  say,  'OK,  what 's  curriculum  and  instruction  doing  about  that? 
What 's  PALMS  doing  about  that?  What 's  Special  Ed  doing  about  that?  What 's  Lunch 
Programs  doing  about  that? '  If  that's  our  priority.  So  I  think,  and  "What's  School  & 
District  Accountability  doing  about  that?  And  by  the  way,  who  are  you  using  to  get  advice  to 
schools  and  districts  about  what  they  should  be  doing  in  their  math  program?  "  Helping  to 
connect  into  this  vast  network  of  professional  development  providers  that  we  've  been 
building  up. 

Because  Kentucky's  education  reform  act  was  a  model  for  MERA,  the  organization  of  its 
Department  of  Education  provides  an  interesting  point  of  comparison.  See  Appendix  1  for 
details. 

Decisionmaking  Processes 

Within  DOE,  according  to  our  interviewees  there,  major  decisions  about  priorities  are  set  by  the 
Board  of  Education  and  by  the  Commissioner  in  meetings  with  senior  staff  members.  In 
interviews,  DOE  senior  staff  tended  to  describe  how  their  work  priorities  are  set  in  terms  of  the 
hierarchy — ^that  they  receive  priorities  from  the  Board  of  Education,  the  Commissioner  and 
Deputy  Commissioners,  or  others  above  them  in  the  formal  organization.  However,  several 
people  also  said  that  the  Commissioner  does  a  good  job  of  getting  information  from  within  the 
organization  and  consulting  with  staff  members  below  the  senior  level  when  it  is  appropriate  to 
do  so. 

Competition  to  set  an  agenda  for  education  and  pressure  on  DOE,  both  from  the  legislature  and 
from  elsewhere  in  the  executive  branch,  have  undermined  the  cohesiveness  of  MERA 
implementation.  Because  DOE  is  part  of  the  executive  branch  of  state  government,  its  work  is 
affected  by  gubernatorial  priorities.  For  example,  in  several  of  our  interviews  we  discussed 
whether  or  not  DOE  had  ever  made  a  case  to  the  legislature  for  fimding  of  more  staff  and 
administrative  capacity  to  implement  Education  Reform.    People  knowledgeable  about  the 
direction  of  DOE  leadership  at  the  time  said  that  it  had  not,  because  of  the  commitment  to 
reduced  regulation  discussed  earlier.  However,  others  said  that  even  if  DOE's  leaders  had 
wanted  more  capacity,  they  would  not  have  been  free  to  make  this  case  to  legislators  because 
expansion  of  state  bureaucracy  was  contrary  to  the  priorities  of  the  Weld  and  Cellucci 
administrations.  Without  a  specific  request  from  the  executive  branch,  the  legislature  would  not 
have  increased  DOE's  budget  on  its  own  because  doing  so  would  make  Democratic  legislative 
leaders  look  like  big  spenders  in  contrast  to  the  fiscal  conservatives  in  the  Governor's  Office. 
Until  1996,  DOE  also  competed  for  scarce  resources  with  the  Executive  Office  for  Educational 
Affairs. 
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Increasing  the  state  role  in  educational  standard  setting,  assessment,  and  accountability  has  made 
education  a  more  visible  policy  issue  for  elected  officials  such  as  the  Governor  and  the  members 
of  the  legislature,  which  has  complicated  agenda-setting  and  decision-making.  With  more 
people  and  agencies  trying  to  participate,  the  possibilities  for  conflict  among  state-level  actors 
have  increased.  This  has  been  most  apparent  in  the  area  of  accoimtability,  where  there  has  been 
tension  within  the  executive  branch  (i.e.  between  the  Governor's  Office  and  the  Department  of 
Education)  over  which  entity  should  have  which  responsibilities. 

Intervention  in  education  policy  by  the  Governor  and  the  legislature  has  sometimes  produced 
instability  in  policy  goals.  A  recent  example  is  the  outside  provisions  to  the  budget  resolution 
passed  in  2000,  which  DOE  staff  see  as  having  imposed  gubernatorial  priorities,  such  as  the 
competency  testing  of  mathematics  teachers  in  schools  with  low  MCAS  scores,  on  them.  The 
legislature  also  responds  to  constituents'  pressure  to  "do  something"  about  various  education 
issues.  As  a  result  of  various  legislative  initiatives,  local  districts  now  have  fifteen  different 
mandated  reporting  requirements.  Interviewees  at  DOE  were  also  concerned  about  the  lack  of 
communication  between  the  department  and  the  rest  of  the  executive  branch  and  between  the 
department  and  the  legislature. 

In  addition  to  outright  conflict,  lack  of  clarity  over  roles  has  sometimes  been  a  problem.  The 
Massachusetts  Business  Alliance  for  Education's  1995  report  on  implementation  of  MERA 
reports  that  confusion  about  the  respective  roles  of  the  Department  of  Education,  the  Board  of 
Education,  the  Executive  Office  of  Educational  Affairs,  the  Office  of  Management  and 
Information  Systems,  MCET,  the  Higher  Education  Coordinating  Council,  and  private  vendors 
was  slowing  the  implementation  of  MERA.  Failure  to  coordinate  weakens  the  comprehensive, 
integrated  intent  of  the  reforms. 

Flow  of  Information  From  DOE  to  the  Field 

In  either  the  "tight-loose"  or  more  interventionist  vision  of  the  state's  role  in  Education  Reform, 
it  is  crucial  that  DOE  communicate  effectively  with  local  educators.  The  formal  and  informal 
channels  through  which  this  communication  takes  place  are  thus  a  crucial  organizational 
element.  In  this  section,  we  emphasize  communication  from  DOE  to  educators  in  the  field;  in 
the  section  of  Part  III  on  collection  and  use  of  data,  we  will  also  address  the  flow  of  information 
in  the  other  direction,  from  the  field  to  DOE. 

Our  survey  of  local  educators  included  several  questions  that  explored  how  they  get  information 
about  Education  Reform,  in  general,  and  from  DOE  in  particular.  The  major  source  of  Education 
Reform  information  is  DOE  itself: 
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Table  2  -  Summary  of  Responses  Regarding  Sources  of  Information  About  Education  Reform 

(Respondents  were  asked  to  give  more  than  one  response  if  they  use  more  than  one  source  of  information.) 


How  do  you  get  information  about  Education  Reform? 

Sup. 

Prin. 

Teach. 

Documents  mailed  directly  to  me  by  the  Department  of  Education 

96.4% 

98.2% 

53.8% 

The  Department  of  Education  web  site 

88.4% 

70.9% 

37.2% 

The  media 

55.4% 

49.7% 

73.1% 

Via  a  union  or  professional  association 

54.5% 

46.7% 

75.6% 

Documents  mailed  by  the  Department  of  Education  to  districts  and  distributed  in 
schools 

NA 

74.5% 

74.4% 

Documents  mailed  directly  to  me  by  other  state  agencies  or  offices 

44.6% 

26.7% 

14.1% 

Several  differences  among  superintendents',  principals',  and  teachers'  responses  bear  noting. 
Superintendents  were  more  likely  to  get  documents  from  state  agencies  or  offices  other  than 
DOE  and  were  more  likely  to  get  information  from  the  Department  of  Education's  web  site  than 
were  either  principals  or  teachers.  Teachers  were  less  likely  than  either  group  of  administrators 
to  get  information  directly  from  the  Department  of  Education  or  from  the  Department's  web  site. 
However,  teachers  were  more  likely  to  use  the  media,  imion  and  professional  associations,  and 
documents  passed  on  at  the  local  level  from  the  state  as  sources  of  information  than  were 
superintendents  and  principals.  This  difference  matters,  since  it  implies  that  DOE's 
communications  with  teachers  are  less  direct  than  with  other  groups  of  educators.  Whether  this 
indirect  communication  is  a  cause  or  an  effect  of  teachers'  seemingly  lower  opinion  of  DOE  and 
Education  Reform  is  a  question  worth  addressing  in  further  research. 

We  also  asked  superintendents  and  principals  how  frequently  they  contact  DOE: 


Table  3  -  Summary  of  Responses  Regarding  Frequency  of  Contact  With  DOE 


How  frequently  do  you  contact  the  Department  of  Education? 

Sup. 

Prin. 

Never 

0.0% 

6.1% 

Once  Per  Year 

0.9% 

11.0% 

Several  Times  Per  Year 

37.8% 

73.8% 

Once  Per  Month 

28.8% 

6.1% 

Once  Per  Week 

22.5% 

2.4% 

More  Than  Once  Per  Week 

5.4% 

0.0% 

This  question  was  not  asked  of  teachers.  Superintendents  were  significantly  more  likely  to 
contact  DOE  than  were  principals.  Interestingly  (see  below),  superintendents  also  tend  to  see 
DOE  staff  in  a  more  positive  light  than  do  principals,  which  again  suggests  that  there  is  a  link 
between  frequency  of  contact  with  DOE  and  positive  opinions  about  DOE  and  its  activities. 

We  also  asked  respondents  how  they  most  frequently  contact  DOE.  Administrators  are  likelier 
than  teachers  to  contact  DOE  and  to  rely  on  existing  contacts  at  the  agency: 
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Table  4-  Sununary  of  Responses  Indicating  How  Respondents  Most  Frequently  Contact  the 
Department  of  Education 

(Some  respondents  chose  more  than  one  response.) 


How  do  you  (or  your  staff)  most  frequently  contact  someone  at  the  Department  of  Education? 

Teachs. 

Admins. 

All 

I  directly  contact  a  specific  department  or  program  area 

19.5% 

55.2% 

47.5% 

I  use  the  website  to  find  the  appropriate  contact  person  and  I  call  him/her 

18.2% 

40.1% 

35.3% 

I  directly  contact  someone  I  already  know 

6.5% 

41.9% 

34.2% 

I  use  the  switchboard  operator  to  direct  me  to  the  appropriate  contact  person 

29.9% 

23.5% 

24.9% 

I  use  the  website  to  find  the  appropriate  contact  person  and  I  send  him/her  an 
email  message 

18.2% 

40.1% 

18.1% 

I  do  not  contact  the  Department  of  Education 

44.2% 

7.2% 

15.3% 

As  DOE  prepares  to  move  many  of  its  information  gathering  and  distribution  functions  onHne,  it 
should  also  take  note  of  teachers'  current  relatively  low  use  of  the  website  as  a  source  of 
information — ^there  may  be  a  need  for  outreach  work  to  increase  educators'  comfort  and 
familiarity  with  the  use  of  this  medium. 

Respondents  indicated  that  they  often  find  it  difficult  to  get  information  from  DOE: 

Table  5  -  Simmiary  of  Responses  Regarding  Ease  of  Obtaining  Information 


How  easy  is  it  to  set  information  from  the  Department  of  Education? 

Sup. 

Prin. 

Teach. 

Very  Easy 

11.2% 

11.5% 

9.2% 

Easy 

62.6% 

50.3% 

46.2% 

Difficult 

15.9% 

26.8% 

24.6% 

Very  Difficult 

1.9% 

4.5% 

15.4% 

There  were  no  differences  between  superintendents  and  principals  on  this  item,  but  teachers  were 
more  likely  to  indicate  that  they  found  it  difficult  to  obtain  information  from  the  Department  of 
Education.  Again,  this  pattem  indicates  that  the  lines  of  communication  between  DOE  and 
teachers  are  more  problematic  than  with  the  other  educator  constituencies.  A  number  of 
comments  from  the  open-ended  questions  also  related  to  flow  of  information.  More  specifically, 
many  respondents  commented  on  how  slow  the  Department  of  Education  is  at  returning  calls  and 
answering  inquiries.    These  issues  may  be  symptoms  of  under-staffing  at  the  Department,  an 
issue  that  will  be  taken  up  in  the  next  section  of  this  report.  Despite  the  generally  negative 
pattem  of  responses,  some  of  the  open-ended  question  responses  noted  that  the  Department  of 
Education  has  greatly  improved  communication  through  better  use  of  technology  and 
information  systems. 

Interviewees  recommended  increasing  the  linkages  between  the  state,  stakeholder  organizations, 
and  local  school  districts.  Survey  respondents  were  also  asked  in  an  open-ended  question  about 
their  recommendations  for  change.  While  several  themes  emerged  in  response  to  this  question, 
many  respondents  emphasized  the  need  for  more  constructive  relationships  between  the  states 
and  local  districts,  including  more  contact  with  and  visits  to  the  field  by  DOE  staff. 
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Conclusion 

DOE  is  not  the  only  state-level  entity  involved  in  implementing  Education  Reform.  Where  other 
actors,  such  as  the  legislature  and  the  Governor,  have  been  involved,  communication  and 
coordination  have  posed  challenges.  Communication  and  coordination  also  appear  to  be 
challenging  within  DOE  and  between  DOE  and  educators  in  the  field.  Many  of  these 
organizational  issues  may  be  the  result  of  shortfalls  in  fmancial  and  human  resources,  which  the 
next  section  of  the  report  will  discuss. 


Financial  and  Human  Resource  Issues 

Outside  of  the  Department  of  Education,  there  are  only  a  few  state  staff  who  work  on  educational 
issues.  For  that  reason,  our  analysis  of  financial  and  human  resources  for  MERA 
implementation  focuses  on  DOE.  We  consider  financial  and  human  resources  in  the  same  section 
because  we  found  such  a  strong  consensus  that  if  DOE  were  to  receive  more  funds  it  should 
spend  them  on  staff 

MERA  added  to  DOE's  responsibilities,  but  did  not  take  any  fimctions  away  fi-om  the 
department.  This  fact,  coupled  with  the  large  decline  in  the  Department's  staffing  between  1980 
and  1993  (from  990  to  325)  and  the  net  increase  of  only  13  FTE  between  1993  and  1998, 
suggests  that  DOE  is  understaffed  to  fiilfill  its  function  under  MERA  as  the  leader  in  Education 
Reform. 

Since  1980,  the  numbers  of  both  state  and  federally  funded  positions  at  DOE  have  declined,  but 
state-funded  positions  have  declined  more.  As  a  result,  DOE  not  only  has  fewer  total  staff  than  it 
did  twenty  years  ago,  but  also  has  a  smaller  proportion  of  them  free  to  work  on  state  rather  than 
federal  programs. 

Federal-  and  State-Funded  DOE  Staff 

Beginning  with  passage  of  the  Elementary  and  Secondary  Education  Act  of  1965,  state  education 
agencies  (SEA's)  across  the  United  States  grew  so  that  they  could  effectively  implement  federal 
policies  (see  Appendix  1  for  more  information).  Between  1965  and  1970,  total  SEA  staff 
nationwide  doubled,  with  nearly  60%  of  the  growth  due  to  federal  programs.  In  addition  to 
compensatory  education,  education  of  students  with  disabilities,  and  bilingual  education,  the 
federal  government  also  had  a  program  (Title  V  of  the  Elementary  and  Secondary  Education 
Act)  specifically  intended  to  increase  SEAs'  capacity  to  implement  federal  programs. 

Figure  1  below  shows  how  the  proportion  of  DOE  FTE's  that  are  federally  fiinded  has  grown 
while  the  overall  number  of  DOE  FTE's  funded  by  both  federal  and  state  sources  has  shrunk. 
The  data  from  which  this  table  was  constructed  appear  in  Appendix  3. 
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Figure  1 

MA  DOE  State  and  Federal  Staffing  Levels 

FY80-FY01 

(excludes  consultants) 
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In  general,  the  potential  problem  with  having  a  large  proportion  of  FTE's  federally  funded  is  that 
their  job  is  to  implement  federal  programs  rather  than  state  priorities.  In  a  few  cases,  federal 
programs  mesh  well  with  MERA.  Some  of  the  DOE  staffers  we  interviewed  described  using  the 
flexibility  Massachusetts  had  as  one  of  the  "Ed  Flex"  demonstration  sites  for  the  federal  Goals 
2000  program  to  enhance  MERA  implementation  capacity.  For  example.  Goals  2000  funds 
supported  districts  producing  mentoring  and  certification  plans  in  districts.  Goals  2000  has  also 
paid  for  professional  development  and  for  the  sabbatical  teachers  who  spend  two  years  at  DOE 
working  with  districts  to  help  implement  the  Curriculum  Frameworks.  Unfortunately,  Goals 
2000  funding  ends  this  year.  Many  other  federal  programs  represent  basic  functions,  such  as 
school  nutrition  programs,  that  neither  conflict  with  MERA  nor  are  central  to  its  implementation. 

In  areas  where  federal  requirements  are  less  flexible  than  in  Goals  2000,  DOE  has  recently  run 
afoul  of  federal  regulations  in  two  areas.  First,  in  some  programs  it  has  failed  to  make  required 
matches  of  federal  fiinds  with  state  fiinds.  Second,  because  of  DOE's  lack  of  administrative 
funds  (see  below),  federal  money  has  sometimes  been  used  to  support  the  department's 
administrative  functions.  Because  this  practice  violates  federal  requirements,  the  state  has  had  to 
reimburse  the  USDOE  (paying  a  penalty  plus  interest)  for  funds  used  in  this  way. 

DOE  as  an  "ATM" 

The  reason  why  DOE's  staff  levels  have  not  risen  much  since  1993,  despite  huge  increases  in 
total  state  spending  on  education,  is  that,  like  other  states,  Massachusetts  "passes  through"  nearly 
all  of  the  state  and  federal  funds  it  receives.  According  to  a  1994  report  by  the  United  States 
General  Accounting  Office,  95%  of  all  federal  money  and  99%  of  all  state  money  received  by 
state  departments  of  education  goes  directly  to  local  schools  and  districts  (GAO,  1994).  On  the 
average,  for  every  $1  retained  by  the  Massachusetts  DOE,  $83  goes  out  to  the  districts.  As  one 
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DOE  senior  staff  member  put  it  in  an  interview,  the  Department  acts  like  an  "ATM"  (automatic 
teller  machine)  for  districts.  After  the  Executive  Office  of  Educational  Affairs  was  eliminated, 
DOE's  budget  increased  slightly,  but  when  the  state  finally  fiinded  the  Massachusetts  Education 
Reform  Review  Commission,  the  money  for  it  was  taken  out  of  the  DOE's  administrative 
budget. 

When  the  legislature  appropriates  funds  for  programs,  it  generally  does  not  set  aside  any  portion 
for  DOE  administration.  According  to  one  of  the  DOE  staff  we  interviewed,  the  DOE 
administration  proposal  included  in  this  year's  House  version  of  the  budget  will  not  even  cover 
rent  on  the  department's  two  buildings.  Another  example  of  a  large  unfunded  administrative 
expense  is  the  printing  and  distribution  of  curriculum  fiameworks  as  they  are  written  and 
revised.  Districts  have  resisted  the  idea  of  downloading  the  documents  firom  the  DOE  website. 

Resource  Shortfalls  at  DOE 

During  2000  and  the  first  part  of  2001,  DOE  was  organized  into  eight  clusters  under  the 
supervision  of  staff  at  the  Associate  Commissioner  level,  two  areas  supervised  by  Deputy 
Commissioners,  and  the  Commissioner's  Office  itself  (see  Figure  2). 


Figure  2 
Simplified  DOE  Organizational  Chart  (prior  to  2001  reorganization) 
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The  bar  graph  below  shows  the  number  of  fall-time  employees  per  DOE  cluster  in  February, 
2000.  The  largest  number  of  full-time  employees  were  in  Accountability,  Education  Program 
Services,  Finance,  and  School  Readiness. 
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Figure  3 
Number  of  FTE's  by  DOE  Cluster 

as  of  February  2000 
(excludes  part-time  employees  and  consvdtants) 
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Nearly  all  of  our  interviewees  inside  and  outside  DOE  agreed  that  the  Department  does  not  have 
the  resources  it  needs  to  implement  Education  Reform  successfully.  The  major  complaint  (both 
from  DOE  people  and  others)  is  about  staffing  levels.  DOE  appears  to  be  understaffed  for  both 
regulatory  compliance  functions  and  support  functions. 

In  regulation,  for  example,  districts'  comphance  with  the  Time  and  Learning  requirements  is  not 
audited  to  ensure  that  reported  "instructional  time"  actually  is  that,  rather  than  study  halls. 
Districts  report  to  DOE  on  whether  they  are  spending  the  required  amount  of  money  in  areas 
such  as  professional  development,  but  DOE  does  not  impose  any  sanctions  or  penalties  on 
districts  that  do  not  meet  these  spending  requirements  (see  State  Auditor's  Report,  1997?). 

DOE  does  not  have  sufficient  staff  to  support  districts'  efforts  to  implement  the  curriculum 
frameworks  (for  the  whole  state,  there  are  two  sabbatical  teachers  at  DOE  offering  support  in  the 
area  of  language  arts).  The  Department  does  not  have  adequate  internal  staff  for  research  and 
evaluation.  It  also  does  not  support  as  broad  a  range  of  technical  assistance  to  underperforming 
schools  as  do  some  other  state  departments.  Professional  development,  a  function  that  falls 
mainly  within  the  Educator  Quality  cluster,  is  another  area  of  major  need.  The  legislature  made 
a  clear  policy  decision — ^based  on  the  "tight-loose"  conception  of  the  state  role  in  Education 
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Reform — ^to  distribute  resources  to  school  districts,  via  the  Chapter  70  funds  earmarked  for 
district  professional  development,  rather  than  build  capacity  at  the  state  level.  This  state-level 
capacity  need  not  have  been  within  DOE  itself;  one  competing  proposal  was  to  create  regional 
professional  development  centers  in  the  state  colleges. 

According  to  the  1 997  Report  on  professional  development  commissioned  by  MERRC  and  a 
study  of  the  implementation  of  the  state  mathematics  framework  published  by  the  Massachusetts 
Teachers'  Association  in  2000,  the  "tight-loose"  assumption  may  not  have  been  warranted  in  this 
area.  Local  districts  and  schools  do  not  believe  they  know  what  to  do  to  raise  all  students' 
achievement  levels,  and  they  resent  the  lack  of  guidance  they  have  received  from  DOE. 

Staffmg  levels  are  a  big  part  of  the  lack  of  DOE  capacity.  In  our  surveys,  local  teachers  and 
administrators  agreed  that  DOE  is  understaffed.  Our  survey  instruments  included  three  questions 
intended  to  assess  local  educators'  experiences  with  DOE  staff.  In  response  to  the  statement, 
"The  Department  of  Education  seems  to  have  enough  staff  to  carry  out  its  responsibihties," 
28.2%  of  respondents  chose  "strongly  disagree,"  30.7%  chose  "disagree,"  12.8%  chose  "agree," 
and  3.9%  chose  "strongly  agree,"  with  the  remainder  selecting  "I  don't  know."  It  is  necessary  to 
note  that  a  larger  proportion  of  individuals  responded  "I  don't  know"  to  the  three  survey  items 
about  DOE  personnel  than  to  any  of  the  other  twenty  similar  items.  Between  20%  and  34%  of 
respondents  chose  that  option  for  these  three  items.  This  may  indicate  that  many  district  and 
school-level  personnel  have  not  had  enough  contact  with  the  DOE  to  judge  its  capacity 
accurately. 

The  surveys  also  included  an  open-ended  question  about  state  support  for  MERA 
implementation:  "Do  you  believe  that  the  Department  of  Education  has  appropriate  human, 
technical,  or  financial  resources  in  order  to  carry  out  its  responsibilities  under  the  Education 
Reform  Act?"  In  responses  to  this  question,  many  respondents  elaborated  upon  the  point  that 
DOE  staff  seem  to  be  working  as  hard  as  they  can  and  that  the  Department  needs  more  staff. 
Another  possibility  is  that  DOE  needs  to  redefine  its  employees'  responsibilities. 

Use  of  Consultants  by  DOE 

When  it  cannot  get  authorization  for  a  new  state  position,  or  when  managers  want  to  go  outside 
normal  state  hiring  procedures  for  some  reason,  DOE  often  hires  consultants  (also  known  as 
"03 's"  after  their  position  classification  number).  At  any  given  time,  there  are  between  60  and  90 
consultants  at  DOE.  The  projects  they  work  on  may  last  several  years  (not  necessarily  with  the 
same  consultants).  Based  on  our  interviews,  some  consultants  are  working  in  positions  that  are 
neither  temporary  nor  peripheral:  a  psychometrician  in  the  assessment  cluster,  many  of  the  staff 
who  process  certification  paperwork,  and  the  people  running  some  of  the  smaller  federal 
programs.  Full  information  from  FY  1995-2001  on  the  numbers  of  consultants,  maximum  and 
minimum  amounts  paid  to  consultants,  and  average  payment  to  consultants  for  each  DOE 
program  area  appears  in  Appendix  6  of  this  report.  We  were  unable  to  obtain  data  on  consultant 
use  from  years  prior  to  FY  1995,  which  hinders  our  abihty  to  compare  trends  in  numbers  of 
consultants  with  trends  in  regular  FTE  staffing  during  the  period  in  which  DOE's  staff  was  cut 
most  and  during  the  first  two  years  of  MERA  implementation. 
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The  chart  below  shows  the  total  numbers  of  consultants  and  regular  FTE  staff  members  at  DOE 
for  fiscal  1995-2000. 

Table  6— Regular  Staff  and  Consultants  at  DOE,  FY  1995-2000 


Fiscal  Year 

FTE  Staff 

Consultants 

1995 

346 

83 

1996 

335 

90 

1997 

338 

82 

1998 

338 

71 

1999 

399 

77 

2000 

395 

61 

The  largest  numbers  of  consultants  have  worked  in  Certification  Administration.  Other  DOE 
areas  hiring  relatively  large  numbers  of  consultants  are  Adult  and  Community  Learning, 
Learning  Support  Services,  Legislative  Affairs,  Math  and  Science,  Operations  Management, 
Reading,  and  Student  Assessment. 

Table  7— Number  of  Consultants  in  DOE  Programs,  FY  1995-2000 


1995 

1996 

1997 

1998 

1999 

2000 

Accountability 

0 

0 

0 

0 

4 

2 

Adult  &  Community  Learning 

7 

5 

9 

8 

4 

4 

Certification 

19 

19 

23 

22 

19 

18 

Charter  Schools 

1 

0 

1 

0 

0 

0 

Chief  Financial  Officer 

0 

0 

1 

0 

0 

0 

Chief  Technology  Officer 

1 

1 

0 

2 

3 

3 

Commissioner's  Office 

1 

0 

3 

2 

7 

5 

Administration  &  Policy 

1 

0 

0 

0 

0 

0 

Early  Childhood 

0 

0 

0 

0 

3 

3 

Education  Program  Services 

4 

0 

0 

0 

1 

0 

Educator  Prep.  In  Higher  Ed. 

0 

4 

3 

2 

1 

1 

Humanities 

2 

0 

0 

0 

0 

0 

Learning  Support  Services 

13 

9 

2 

6 

8 

7 

Legal  Office 

1 

1 

2 

1 

1 

0 

Legislative  Affairs 

0 

7 

6 

4 

2 

2 

Mathematics  and  Science 

4 

7 

5 

7 

7 

7 

Media  Relations 

1 

2 

1 

0 

1 

0 

Nutrition 

1 

1 

0 

1 

1 

0 

Operations  Management 

0 

8 

10 

8 

8 

4 

Program  Quality  Assurance 

1 

1 

1 

0 

2 

2 

Reading 

14 

9 

4 

1 

1 

0 

School  Business 

1 

2 

2 

2 

2 

1 

School  to  Career 

1 

2 

0 

0 

0 

0 

Special  Education  Appeals 

3 

5 

5 

3 

0 

0 

Special  Services 

1 

0 

0 

0 

0 

0 

Student  Assessment 

6 

7 

4 

2 

2 

2 

TOTAL 

83 

90 

82 

71 

77 

61 
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DOE  staff  saw  some  advantages  to  using  consultants.  Hiring  people  as  consultants  rather  than  as 
state  employees  allows  DOE  to  offer  more  money  to  qualified  staff  than  the  state  salary  scale 
permits.  Also,  the  use  of  consultants  can  allow  DOE  to  respond  more  quickly  to  programmatic 
needs  because  it  is  quicker  to  hire  a  consultant  than  to  create  a  new  state  position. 

For  several  reasons,  it  is  difficult  to  tell  whether  use  of  consultants  during  the  implementation  of 
Education  Reform  compensated  for  the  regular  DOE  staff  lost  during  the  1980s  and  early  1990s. 
First,  we  do  not  have  comparably  detailed  information  on  regular  staff  expenditures.  Second, 
although  we  have  data  on  how  many  consultants  each  area  of  DOE  used  and  how  much  each 
consultant  was  paid,  we  do  not  know  how  many  hours  each  person  worked  or  what  they  did. 
Third,  we  do  not  know  whether  use  of  consultants  increased  or  decreased  during  the  years  when 
DOE's  regular  staff  was  severely  cut,  so  we  cannot  precisely  determine  the  extent  to  which 
consultant  use  compensated  for  lost  positions.  Further  study  is  needed  of  these  important  issues, 
in  the  context  of  DOE's  overall  staffing  levels  and  patterns. 

Based  on  the  available  data,  however,  we  suspect  that  the  consultants  did  not  make  up  for  a  lack 
of  professional  staff  First,  although  we  do  not  have  data  on  the  number  of  hours  each  consultant 
worked,  we  surmise  fi:om  the  amounts  they  were  paid  (see  Appendix  6)  that  many  if  not  most 
worked  less  than  the  equivalent  of  fiiU  time  for  DOE,  so  it  is  generally  not  fair  to  say  that  one 
consultant  replaces  one  FTE.  Second,  the  data  in  Table  7  show  that  in  any  given  year,  between 
21%  and  31%  of  all  consultants  working  at  DOE  were  in  Certification  Administration,  and  we 
know  from  interviews  that  these  people's  role  was  processing  paperwork — a  crucial  Department 
function,  but  not  one  of  the  ones  such  as  assessment  and  accountability  in  which  DOE  has  faced 
enormous  challenges  in  MERA  implementation. 

Even  if  use  of  consultants  had  fully  filled  the  gaps  left  by  cuts  in  regular  DOE  staffing,  other 
factors  make  reliance  on  consultants  only  a  second-best  strategy.  Over  time,  heavy  use  of 
consultants  creates  two  tiers  of  staff  earning  different  salaries  and  benefits  (no  benefits,  in  the 
case  of  the  consultants).  More  problematically  from  the  standpoint  of  comprehensive,  long-term 
education  reform,  it  often  means  more  staff  turnover.  Consultants  are  generally  either  people 
seeking  flexible  work  or  people  who  would  rather  be  in  positions  with  fiill  benefits,  and  in  either 
case  they  are  likely  to  change  jobs  frequently.  As  one  interviewee  outside  of  DOE  commented, 
the  most  important  issue  may  not  be  whether  use  of  consultants  is  good  or  bad  per  se,  but  that  the 
DOE  lacks  an  overall  plan  for  human  resources  with  reference  to  which  it  can  defend  its  use  of 
consultants. 

Qualifications  of  DOE  Staff 

Many  educators  in  the  field  perceive  DOE  staff  as  lacking  appropriate  qualifications  or  education 
experience.  In  our  survey,  nearly  half  of  the  respondents  who  answered  the  item  about  whether 
"Department  of  Education  staff  members  have  the  qualifications  and  experiences  they  need  to 
provide  useful  support  to  schools  and  districts"  chose  "strongly  agree"  or  "disagree,"  compared 
with  18.1%  who  chose  "agree"  or  "strongly  agree."  The  remainder  chose  "I  don't  know,"  with 
teachers  much  more  likely  than  principals  or  superintendents  to  choose  this  option.  Among 
those  who  did  answer,  administrators  were  almost  twice  as  likely  as  teachers  to  give  favorable 
responses,  and  superintendents  were  more  likely  than  principals  to  view  DOE  staff  qualifications 
favorably. 
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There  are  many  positions  at  DOE,  such  as  in  financial  management,  law,  and  public  relations,  for 
which  school-based  experience  does  not  seem  to  be  especially  important.  It  is  also  not  accurate 
to  say  that  DOE  senior  staff  generally  lack  school-based  experience.  A  few  (including  the 
Commissioner)  have  extensive  experience  as  teachers  or  administrators,  and  most  have  done  at 
least  some  teaching.  However,  because  at  the  most  senior  levels  DOE  staff  are  likely  to  have 
been  with  the  agency  for  a  significant  portion  of  their  careers,  their  field  experience  is  not  recent. 
Also,  some  of  the  highest-profile  senior  staff  have  the  least  district-  or  school-based  experience. 

In  the  positions  for  which  school-based  experience  is  important,  one  obstacle  to  bringing 
educators  into  DOE  staff  positions  is  that  the  state  employee  pay  scale  is  generally  lower,  for 
twelve-month  contracts  than  what  experienced  teachers,  and  especially  administrators,  can  earn 
in  the  field  (often  on  nine-  or  ten-month  contracts).  Figure  4  shows  this  gap. 


Figure  4 
Average  Annual  Salary  Estimates  Local 

Schools/Districts  v.  Dept.  of  Education 

(FY  2000) 
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Sources:  Massachusetts  Teachers  Association,  Massachusetts  Association  of  Secondary  School  Administrators, 
Massachusetts  Association  of  School  Superintendents,  Massachusetts  Department  of  Education 


Because  DOE  does  not  have  data  available  on  salaries  of  teachers  or  administrators,  the  averages 
in  Figure  4  come  from  information  provided  by  teacher  and  administrator  associations. 
According  to  the  Service  Employees  International  Union  (SEIU),  which  represents  the  bulk  of 
DOE's  professional  employees,  most  of  the  Education  Specialists  are  at  Levels  III  and  FV.  The 
Program  Coordinators  whose  salaries  were  used  to  calculate  the  averages  in  the  chart  are  in 
supervisory,  high  technology,  program  coordination,  or  professional  positions,  identified  by  their 
membership  in  Unit  6  of  the  National  Association  of  Government  Employees  (NAGE).  The 
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average  management  salaries  were  calculated  based  on  salaries  for  fiscal  2000. '  The  salary  gaps 
shown  in  the  chart  are  a  major  reason  why  there  are  not  more  experienced  teachers  and 
administrators  working  at  DOE.  For  administrators,  shortages  of  principals  and  superintendents 
in  Massachusetts  and  elsewhere  are  probably  also  a  factor. 


Specific  Areas  of  Capacity  and  Resource  Needs  at  DOE 

In  our  interviews  with  a  wide  cross-section  of  DOE  staff  and  other  state-level  policy  makers  and 
stakeholders,  several  key  areas  emerged  in  which  DOE  particularly  needs  resources  or  additional 
capacity  to  carry  out  its  MERA  responsibilities. 

The  first  major  area  of  need,  accountability,  goes  beyond  DOE  and  includes  the  new 
accountability  function  of  the  Governor's  Office.  For  MERA  to  work  as  intended,  the  state 
needs  to  have  the  capacity  to  hold  all  schools  and  districts  accountable  for  improved  student 
performance,  rather  than  only  identifying  and  intervening  in  a  few.  Staff  is  a  big  part  of  the 
need,  but  so  is  fimding  for  data  collection  and  potential  contracting  out  of  some  evaluation 
functions. 

Overall,  data  collection  is  the  second  area  of  need.  Ever  since  the  begiiming  of  Education 
Reform,  capacity  for  data  collection  and  management  has  lagged  behind  what  is  needed  to 
monitor  districts'  compliance  with  the  law  and  progress  towards  implementation.  The  state  will 
soon  launch  its  online  "data  warehouse"  of  student  and  teacher  information  (including  test 
scores),  called  Data  Mart,  but  online  data  collection  will  continue  to  be  an  area  of  major  resource 
demands. 

Data  collection  is  related  to  a  third  major  area  of  need,  research  and  evaluation.  DOE  lacks  both 
the  staff  to  do  major  evaluations  and  the  funds  to  contract  with  external  providers  for  evaluation 
and  research  work.  MERRC  has  the  legal  authority  to  conduct  research  and  evaluation,  but  has 
not  had  the  necessary  resources  over  the  whole  course  of  Education  Reform.  If  DOE  is  to 
function  as  a  learning  organization  and  to  guide  local  implementation  of  Education  Reform 
effectively,  greater  capacity  in  these  areas  is  crucial. 

Other  areas  of  need  cited  by  DOE  staff  are  the  School  Building  Assistance  program,  assessment, 
content  area  expertise,  and  bilingual  education.  Some  of  these  needs  might  be  filled  through 
means  other  than  increasing  staff  at  DOE  itself.  For  example,  several  state  leaders  believe  that 
the  School  Building  Assistance  program  should  draw  upon  the  expertise  of  other  executive- 
branch  agencies  that  do  similar  work.  Content  area  expertise,  and  assistance  in  producing  test 
items  for  MCAS,  might  come  from  field-based  educators  on  contract  to  DOE. 


'  The  average  teacher  salary  was  reported  by  the  Massachusetts  Teachers'  Association  based  on  information  from 
the  1999-2000  school  year.  The  average  middle  school  and  high  school  principal  salaries  for  1999-2000  are 
estimates  provided  by  the  Massachusetts  Association  of  Secondary  School  Administrators  (a  precise  figure  was 
unavailable).  The  average  superintendent's  salary  is  based  on  information  reported  by  the  Massachusetts 
Association  of  School  Superintendents  for  the  1999-2000  school  year.  All  of  the  DOE  salary  averages  are  based  on 
the  average  of  FY  2000  actual  salaries  reported  by  DOE. 
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For  the  staff  who  are  currently  at  DOE,  Httle  if  any  professional  development  is  available,  and 
because  of  demands  on  their  time  it  is  difficult  for  them  to  take  advantage  of  the  opportunities 
that  do  exist  through  the  state  Human  Resources  Department.  Senior  staff  would  like  to  have 
professional  development  opportunities  that  would  both  keep  their  staff  intellectually  engaged 
with  their  fields,  and  to  build  their  skills  in  areas  like  staff  management  and  conflict  mediation. 
Several  commented  that  the  lack  of  training  in  administration  is  a  more  serious  problem  for  the 
department  as  a  whole  than  the  lack  of  formal  training  in  content  matters.  There  is  also  a  need  to 
make  sure  that  employees  who  deal  with  schools  and  the  public  have  solid  understanding  of 
MERA,  particularly  given  rising  controversy  over  MCAS. 


Conclusion 

The  major  capacity  challenge  facing  the  Department  of  Education  is  inadequate  numbers  of  staff. 
The  high  proportion  of  DOE  staff  who  are  in  federally  funded  positions  also  reduces  the 
department's  flexibility  in  responding  to  state  policy  priorities.  There  is  particular  need  for  staff 
in  data  collection  and  management,  school  building  assistance,  assessment,  professional 
development,  and  curriculum  content  areas.  DOE's  use  of  consultants  has  both  allowed  greater 
flexibility  in  hiring  and  posed  problems  of  equity  and  staff  turnover;  use  of  consultants  has 
dropped  as  regular  staff  has  grown  sHghtly.  Educators  in  the  field  say  that  DOE  needs  more  staff 
with  extensive  experience  in  schools  and  districts,  but  the  state  pay  scale  poses  difficulties  in 
recruiting  such  people. 

In  Part  ni  of  the  report,  we  will  survey  the  state's  MERA  responsibilities  in  ten  key  areas  and 
examine  how  political,  organizational,  and  resource  issues  have  affected  the  state's 
implementation  capacity. 
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III.     state  Capacity  in  Specific  Areas 


At  the  beginning  of  our  research  process,  we  closely  examined  the  duties  assigned  to  the  state  by 
the  Education  Reform  Act  and  separated  them  into  ten  categories: 

1)  Developing  and  implementing  standards  of  learning; 

2)  Developing  and  administering  a  system  of  student  assessment; 

3)  Developing  an  accountability  system  for  school  and  district  performance; 

4)  Supporting  and  changing  local  governance  and  management  of  pubUc  education; 

5)  Making  effective  state  level  policy  decisions; 

6)  Collecting  and  using  information  to  inform  policy  decisions  and  improve  the  performance  of 
the  state's  educational  system; 

7)  Enhancing  educator  quality  and  education  as  a  profession  through  new  processes  for 
certification  and  professional  development; 

8)  Ensuring  readiness  to  learn  through  early  childhood  and  literacy  programs; 

9)  Encouraging  school  choice  by  requiring  all  districts  to  accept  students  from  other  districts 
unless  their  School  Committee  specifically  voted  not  to  and  creating  charter  schools; 

10)  MloczAmg  funding  in  a  way  that  is  adequate  and  equitable,  both  within  and  across  districts. 

In  this  section,  we  report  on  the  state's  capacity  to  implement  Education  Reform  in  each  of  these 
ten  areas.  In  interviews,  it  was  often  difficult  for  our  respondents  to  separate  their  perceptions  of 
the  state's  capacity  firom  their  overall  evaluation  of  the  pohcies  the  state  was  implementing.  Our 
analysis  focuses  as  much  as  possible  on  capacity  issues  rather  than  on  the  policies  themselves. 
To  be  fair  to  those  involved  in  implementing  Education  Reform,  it  should  be  noted  that  many  of 
the  issues  we  point  out  in  this  section  of  the  report  are  much  clearer  in  hindsight  than  they  would 
have  been  to  decision  makers  and  state  officials  in  the  early  years  of  MERA  implementation.  All 
citations  to  the  law  are  to  the  Massachusetts  Education  Reform  Act,  Chapter  70  of  the  Acts  and 
Resolves  of  1993. 


1 .  Developing  and  Implementing  Standards  of  Leaming 

Section  29  of  MERA  (Section  ID  of  M.G.L.  Chapter  69),  among  many  other  provisions, 
authorizes  the  Board  of  Education  to  direct  the  Commissioner  of  Education  to  institute  a  process 
for  producing  curriculum  firameworks  in  the  core  subjects  (mathematics,  science  and  technology, 
history  and  social  sciences,  English,  foreign  languages,  and  the  arts).  Producing  the  curriculum 
frameworks,  however,  was  only  a  small  portion  of  the  work  that  needed  to  be  done  in  order  to 
implement  them.  Implementing  high  standards  for  all  students  was  related  to  many  other  MERA 
tasks,  such  as  increasing  instructional  time,  integrating  academic  and  vocational  education, 
eliminating  the  general  track  in  high  schools,  and  reviewing  programs  in  bilingual  and  special 
education. 
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Producing  the  Standards 

Although  the  law  did  not  specifically  mandate  a  Common  Core  of  Learning,  the  Department  of 
Education  began  by  producing  one.  According  to  the  DOE's  1993  Implementation  Plan  for 
Education  Reform,  the  Educational  Improvement  team  had  responsibility  for  development  of  the 
interdisciplinary  Common  Core  of  Learning.  Responsibility  for  the  Curriculum  Frameworks, 
which  would  be  written  after  the  Common  Core  was  in  place,  lay  with  the  Instructional  and 
Curriculum  Services  team  and  the  Partnerships  Advancing  Learning  in  Mathematics  and  Science 
(PALMS)  project. 

In  order  to  produce  standards,  DOE  needed  access  to  content-area  expertise.  DOE-based  staff 
drew  upon  a  broad  range  of  expertise  in  producing  the  standards.  The  Common  Core  of 
Learning  process  and  the  early  stages  of  the  production  of  the  Frameworks  involved  a  large 
number  of  local  educators  teaching  in  each  area.  DOE  also  has  had  Sabbatical  Teachers  working 
in  various  content  areas,  funded  by  the  federal  Goals  2000  program.  According  to  one  DOE  staff 
member  who  has  been  involved  with  the  standards,  typically  two  or  three  DOE  staff  work  with  a 
panel  of  between  12  and  20  educators  for  about  eighteen  months  to  produce  each  subject  area 
fi-amework.  MERA  directs  the  Commissioner  (and  by  extension,  the  Department)  to  construct 
standards  "with  due  regard  to  the  work  and  recommendations  of  national  organizations."  The 
process  drew  upon  standard-setting  work  being  done  by  these  associations,  such  as  the  National 
Council  of  Teachers  of  Mathematics. 

For  some  of  our  respondents,  the  drafting  of  the  Common  Core  and  the  initial  process  for 
developing  Curriculum  Frameworks  demonstrates  the  power  of  "distributed  systems"  for  policy 
implementation.  From  this  perspective,  because  so  many  educators  participated  in  producing  the 
Common  Core  (and  the  earlier  drafts  of  the  Curriculum  Frameworks),  they  were  excited  about 
putting  them  to  use  in  their  schools  and  classrooms.  Involving  so  many  people  in  effect  built 
extra  capacity  for  the  state,  outside  of  its  formal  institutional  structures.  Admirers  of  the  early 
process  contrast  it  with  the  more  centralized  process  by  which  the  Frameworks  were  revised 
during  John  Silber's  term  as  Chair  of  the  Board  of  Education.  They  claim  that  the  more 
centralized  approach  taken  by  Silber  requires  more  formal  capacity  than  the  participatory 
approach,  such  as  internal  content  expertise  and  more  staff  for  communicating  the  standards  to 
educators  who  are  unfamiliar  with  (or  in  some  cases,  opposed  to)  them. 

Implementing  the  Standards 

Even  more  challenging  than  developing  standards  is  ensuring  that  they  are  actually  implemented 
by  local  schools  and  districts.  To  get  a  sense  of  the  magnitude  of  this  task,  consider  that  there 
are  371  school  districts,  1,874  schools,  and  71,900  teachers  in  the  Commonwealth.    Early  in 
Education  Reform,  state-level  leaders  assumed  that  local  educators  would  be  able  to  make 
effective  implementation  decisions  without  a  great  deal  of  state  intervention.  However,  in 
practice  this  assumption  has  not  been  uniformly  borne  out  across  the  state. 

In  our  survey,  30.9%  of  respondents  said  the  state  has  done  a  "good"  job  of  supporting  schools 
and  districts  in  aligning  curricula  with  the  state  firameworks.  An  additional  5.9%  said  support 
has  been  "excellent,"  while  27.5%  said  "fair"  and  20.5%  said  "poor."  The  larger  districts  have 
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curriculum  coordinators  who  can  lead  alignment  of  curricula  with  the  Frameworks,  but  the 
smaller  ones  do  not.  DOE  has  supported  implementation  through  professional  development 
programs  such  as  the  summer  Content  Institutes  for  teachers.  It  has  also  used  the  Goals  2000- 
fimded  Sabbatical  Teachers  to  work  with  local  educators.  However,  the  Sabbatical  Teachers 
have  been  stretched  rather  thin.  For  example,  in  the  area  of  English  Language  Arts  only  two 
Sabbatical  Teachers  have  provided  advice  on  implementation  of  the  Curriculum  Frameworks  to 
districts  statewide. 

One  largely  untapped  potential  source  of  support  in  content  areas  is  the  state's  higher  education 
system.  Although  there  is  considerable  interest  and  concern  on  the  part  of  higher  education 
leaders,  engaging  a  wide  range  of  faculty  may  be  challenging.  For  many  higher  education 
faculty  in  arts  and  sciences  (i.e.  outside  schools  of  education),  teacher  preparation  and 
professional  development  are  peripheral  concerns. 

A  longer-term  source  of  fiinding  for  the  Sabbatical  Teachers  program  (ideally,  an  increase  in 
funding  as  well)  is  necessary  if  the  program  is  to  continue  supporting  implementation  of  the 
Curriculum  Frameworks.  The  Goals  2000  money  that  has  been  supporting  the  program  is 
scheduled  to  run  out  later  this  year  because  the  federal  program  has  been  terminated. 


2.  Developing  and  Implementing  a  System  of  Student  Assessment 

MERA  has  two  specific  provisions  that  require  the  state  to  assess  what  students  have  learned. 
Section  29  (M.G.L.  Chapter  69,  Section  II)  requires  the  board  to  adopt  a  criterion-referenced 
assessment^  "designed  both  to  measure  outcomes  and  results  regarding  student  performance,  and 
to  improve  the  effectiveness  of  curriculum  and  instruction."  This  assessment  is  to  be  used  as  the 
basis  of  the  system  for  evaluating  school  and  district  performance.  The  law  also  requires 
"comprehensive  diagnostic  assessment  of  individual  students"  to  be  conducted  "at  least  in  the 
fourth,  eighth,  and  tenth  grades."  These  diagnostic  assessments  "shall  identify  academic 
achievement  levels  of  all  students  in  order  to  inform  teachers,  parents,  administrators  and  the 
students  themselves,  as  to  individual  academic  performance." 

DOE's  1993  MERA  implementation  plan  assigned  the  task  of  "developing  and  implementing  a 
system  of  student  assessment"  to  the  Accountability  and  Evaluation  Services  Team.  Although 
the  law  implies  that  test  used  in  the  school  and  district  evaluation  might  be  different  from  the  one 
used  in  diagnostic  assessment  of  individual  students,  Massachusetts,  like  other  states,  uses  a 
single  assessment  for  both  purposes.  This  test  is  the  Massachusetts  Comprehensive  Assessment 
System  (MCAS).  DOE's  1993  MERA  implementation  plan  assigned  the  task  of  "developing  and 
implementing  a  system  of  student  assessment"  to  the  Accountability  and  Evaluation  Services 
Team. 

Developing  a  standardized  assessment  is  an  enormous  job,  requiring  both  content  expertise  and 
knowledge  of  the  principles  of  psychometrics,  the  science  of  testing.  The  assessment  staff  at 


A  criterion-referenced  assessment  is  one  in  which  student  performance  is  evaluated  against  an  absolute  standard,  in 
contrast  to  a  norm-referenced  assessment,  which  ranks  students'  performances  in  comparison  to  each  other. 
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DOE  is  relatively  small,  but  has  expanded  its  capacity  by  using  contractors  to  produce  the 
MCAS. 

Apart  from  developing  the  tests,  many  other  functions  are  part  of  running  the  assessment 
program.  Scoring  the  tests  is  a  massive  undertaking,  in  particular  the  essay  and  problem-solving 
components.  This  year's  administration  of  MCAS  produced  ten  million  written  responses  from 
students.  DOE  staff  are  responsible  for  managing  the  assessment  program,  conducting  outreach 
to  parents  and  educators,  supervising  the  contractor's  work,  and  producing  numerous  reports  and 
publications  (300  in  the  past  year).  Determining  what  accommodations  to  provide  for  students 
with  disabilities  and  other  special  populations  is  also  a  responsibility  of  the  Department  and 
Board  of  Education. 

Most  people  believe  that  MCAS  is  technically  among  the  very  best  standards-based  state  tests 
and  that  the  test  itself  has  been  well-implemented.  However,  some  believe  that  the  state  has 
failed  to  "employ  a  variety  of  assessment  instruments"  and  "include  consideration  of  work 
samples,  projects  and  portfoHos,  and...  facilitate  authentic  and  direct  gauges  of  student 
performance"  as  Section  29  of  MERA  (M.G.L.  Chapter  69,  Section  II)  requires.  To  some 
extent,  this  can  be  seen  as  a  capacity  issue,  since  this  sort  of  assessment  would  almost  certainly 
require  either  more  staff  for  DOE  or  more  funds  to  use  in  contracting  for  outside  help.  However, 
capacity  is  not  the  only  issue — the  Board  of  Education  made  a  decision  to  employ  only  a 
standardized  assessment,  which  many  experts  believe  is  more  valid  and  reliable  than  assessment 
of  portfolios  and  projects. 

In  our  interviews  and  on  our  survey  we  heard  criticisms  of  how  other  elements  of  assessment 
policy  have  been  implemented.  One  problem  is  the  lack  of  attention  to  testing  issues  for  special 
populations,  and  the  Board  of  Education's  delay  in  putting  a  policy  in  place  on  retesting  for 
students  who  fail  the  tenth-grade  MCAS.  It  seems  that  many  leaders  in  the  state  did  not  grasp  at 
the  outset  how  large  the  challenges  in  these  areas  would  be.  Both  omissions  seem  to  have 
reduced  the  field's  confidence  in  state  policy  makers.  Getting  student  assessment  results  to 
schools  and  districts  in  a  timely  manner  has  proved  extremely  challenging  for  the  state's 
contractors  and  for  DOE  staff,  as  has  assisting  local  educators  in  understanding  what  the  results 
mean. 

In  the  area  of  "interpreting  MCAS  results,"  39.8%  of  local  educators  rated  the  state's  support  as 
"good"  or  "excellent,"  56.7%  as  "fair"  or  "poor,"  and  the  remainder  didn't  know.  In  our  survey, 
66.7%  of  respondents  rated  the  state's  support  of  local  efforts  at  "providing  support  for  students 
who  score  poorly  on  MCAS"  as  "poor"  or  "fair."  29.7%  rated  the  state's  support  in  that  area  as 
"good"  or  "excellent,"  and  the  remainder  chose  "don't  know." 

The  main  reason  why  these  functions  have  not  been  carried  out  as  well  as  DOE  staff  and  state 
policymakers  might  like  is  a  lack  of  staff    Even  though  the  assessment  team  at  DOE  has  been 
relatively  well-funded,  the  office  only  had  eleven  full-time  staff  and  two  consultants  in  FY  2000. 
Areas  of  current  need  for  staff  include  coordination  of  communications  and  publications,  testing 
of  special  populations  such  as  students  with  limited  English  proficiency  and  students  with 
disabilities,  and  an  in-house  psychometrician  on  regular  staff  (currently,  the  staff 
psychometrician  is  a  consultant).  Several  interviewees  noted  that  even  though  a  great  deal  of  the 
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program  is  carried  out  by  a  contractor,  DOE  needs  sufficient  assessment  staff  to  ensure  that  the 
contractor  continues  to  act  in  the  best  interest  of  Massachusetts  students. 


3.  Developing  an  Accountability  System  for  School  and  District 
Performance 

Section  29  of  MERA  (M.G.L.  Chapter  69,  Section  II)  directs  the  Board  of  Education  to  "adopt  a 
system  for  evaluating  on  an  annual  basis  the  performance  of  both  public  school  districts  and 
individual  public  schools"  and  declares,  "The  commissioner  is  autiiorized  and  directed  to  gather 
information,  including  the  information  specified  herein  [the  assessment  system]  and  such  other 
information  as  the  board  shall  require,  for  the  purposes  of  evaluating  individual  public  schools, 
school  districts,  and  the  efficacy  and  equity  of  state  and  federal  mandated  programs." 

As  the  name  of  the  relevant  DOE  cluster.  Accountability  and  Targeted  Assistance,  suggests, 
there  are  two  components  to  an  accountability  system — ^how  districts  are  identified  as  in  need  of 
improvement,  and  how  districts  with  problems  are  given  assistance  to  overcome  them.  Precisely 
which  parts  of  the  state  government  have  authority  to  perform  each  of  these  tasks,  and  what  the 
tasks  entail,  has  been  somewhat  unsettled  for  the  past  several  years. 

Accountability 

In  the  area  of  accountability,  there  have  been  capacity  problems  caused  by  both  insufficient 
resources  and  unclear  organizational  lines.  Because  of  limited  resources,  the  state  accountability 
system  emphasizes  oversight  of  a  few  schools  that  appear  to  have  the  most  egregious  problems, 
rather  than  the  Education  Reform  principle  of  continuous  improvement  for  all  schools  in  the 
Commonwealth.  Compounding  the  resource  problem,  several  state-level  organizations  have 
claimed  roles  in  accountability,  producing  overlapping  mandates  and  inter-agency  conflict. 

Also,  because  Section  29  of  MERA  (M.G.L.  Chapter  69,  Section  IJ)  requires  that  the 
accountability  determinations  be  based  on  the  results  of  the  state  student  assessment,  delays  in 
implementing  MCAS  produced  delays  in  the  accountability  system. 

Section  29  of  MERA  (M.G.L.  Chapter  69,  Sections  IJ  and  IK)  assigns  accoimtability  functions 
to  the  Board  of  Education  and  the  Commissioner  of  Education.  The  Board  is  to  "establish 
regulations  defining  when  a  school  or  school  district  has  chronically  failed  to  improve  the 
educational  program  provided  to  students  served  by  the  school  or  district,"  approve  districts' 
remedial  plans  for  under-performing  schools,  declare  a  school  to  be  "chronically  under- 
performing"  if  it  does  not  demonstrate  significant  improvement  according  to  its  remedial  plan 
within  24  months  after  the  plan's  approval,  and  designate  a  receiver  to  take  a  district  over  when 
it  is  chronically  under-performing.    MERA  directs  the  Commissioner  to  "appoint  an  independent 
fact-finding  team  which  shall  forthwith  assess  the  reasons  for  the  under-performance  and  the 
prospects  for  improvement"  when  a  school  or  district  is  identified  as  under-performing  or  has 
consistently  failed  to  improve  student  performance.  If  the  Board  of  Education  determines  that  a 
district  is  "chronically  under-performing  for  fiscal  reasons,"  then  the  Commissioner  of  Revenue 
becomes  involved.  He  or  she  "shall  determine  the  amount  of  any  deficiency  pursuant  to  the 
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sums  required  under  chapter  seventy,  if  any,  and  issue  an  order  compelling  the  community  to 
provide  a  sum  of  money  equal  to  such  deficiency." 

In  the  1993  implementation  plan,  the  task  of  developing  a  "process  and  regulations  to  identify, 
assist,  and  intervene  in  underperforming  schools,  districts,  and  municipalities"  was  assigned  to 
the  Deputy  Commissioner  and  the  DOE  Legal  Counsel.  Largely  because  MCAS  was  not 
administered  until  1998,  the  Board  of  Education  did  not  approve  accountability  regulations  until 
late  in  1999.  Consequently,  the  most  serious  accountability  sanctions  included  in  MERA  have 
not  yet  been  used.  We  can,  however,  identify  a  serious  capacity  issue  in  the  uncertainty  over 
which  entity  within  the  state  government  would  take  charge  of  holding  schools  and  districts 
accountable.  This  uncertainty  has  compounded  the  delay  in  implementing  the  accountability  and 
targeted  assistance  functions. 

Between  1997  and  2000,  four  different  state  entities  (the  State  Auditor,  the  Education 
Management  and  Accountability  Board,  the  Program  Quality  Assurance  office  within  DOE,  and 
the  Board  of  Education)  took  responsibility  for  evaluating  district  compliance  with  MERA.  The 
first  state  agencies  to  take  a  role  in  holding  districts  accountable  specifically  for  the  goals  of 
Education  Reform  were  outside  the  DOE,  before  the  MCAS  was  in  place  to  serve  as  the 
cornerstone  of  the  Board  of  Education's  accountabilify  regulations.  In  1997,  the  State  Auditor 
produced  critical  audits  of  Education  Reform  as  a  whole,  and  of  the  Lawrence  Public  Schools  in 
particular.  The  Boston  Globe  reported  on  questionable  spending  by  school  officials  in  Lawrence, 
in  response  to  which  Education  Commissioner  Robert  Antonucci  asked  the  Auditor  to 
investigate.  Spurred  by  revelations  that  large  sums  of  Education  Reform  money  had  been  spent 
inappropriately  in  Lawrence,  Governor  William  Weld  created  the  Education  Management  and 
Accountability  Board  (EMAB)  by  executive  order  in  1998.  EMAB  reviewed  the  implementation 
of  Education  Reform  in  22  districts  and  produced  both  individual  district  reports  and  an 
aggregate  report  on  the  first  19  audits.  DOE's  Accountability  and  Targeted  Assistance  group 
collaborated  with  EMAB  on  several  of  those  audits. 

Prior  to  Education  Reform,  DOE  aheady  had  an  accountability  office  of  sorts  in  the  Program 
Quality  Assurance  cluster  (PQA).  The  role  of  PQA  is  to  monitor  local  compliance  with  state  and 
federal  regulations  in  areas  such  as  bilingual  education  and  special  education.  PQA  did  not 
initially  have  a  role  in  monitoring  Education  Reform,  but  the  PQA  office  developed  a  protocol 
for  including  MERA  requirements  in  their  Consolidated  Program  Reviews  and  piloted  it  during 
the  1999-2000  academic  year. 

Late  in  1999,  the  Board  of  Education  produced  its  regulations  and  the  Accountability  and 
Targeted  Assistzmce  cluster  of  DOE  began  preparing  to  evaluate  districts  identified  as 
underperforming  because  of  low  and  stagnant  MCAS  scores.  Some  critics  claim  that  the  Board's 
regulations  rely  too  heavily  on  MCAS,  which  is  designed  to  assess  the  learning  of  individual 
students  rather  than  the  performance  of  schools.  One  alternative  would  be  a  different  sort  of 
assessment,  specifically  designed  to  measure  school  performance.  Maryland  uses  such  a  test 
(MSPAP)  at  the  elementary  school  level.  Another  would  be  a  system  of  evaluation  for  all 
schools,  including  visits  by  evaluation  teams.  For  more  information  on  one  alternative  model,  see 
the  section  of  Appendix  1  on  Rhode  Island's  accountability  system. 
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In  2000,  Governor  Cellucci  included  a  provision  in  his  budget  proposal  that  would  have  shifted 
the  entire  accountability  function  to  a  new  board  to  be  located  in  the  Governor's  Office,  which 
cast  uncertainty  over  the  work  of  DOE's  accountability  staff.  In  the  spring  of  2001,  the  situation 
was  resolved.  The  new  Education  Management  Audit  Council,  chaired  by  James  Peyser  (also 
the  Board  of  Education  chair  and  education  advisor  to  Acting  Governor  Jane  Swift)  and  housed 
in  the  Governor's  Office,  will  take  charge  of  the  "accountability"  function  of  Education  Reform. 
The  Accountability  and  Targeted  Assistance  cluster  of  DOE  will  take  charge  of  assisting 
underperforming  schools,  and  PQA,  which  is  within  Accountability  and  Targeted  Assistance, 
will  continue  the  regulatory  work  it  was  already  doing  but  will  no  longer  monitor  compliance 
with  Education  Reform. 

Targeted  Assistance 

According  to  MERA  (Section  29;  M.G.L.  Chapter  69,  Section  1  J),  the  Commissioner  of 
Education  is  to  provide  "technical  assistance  for  the  improvement  of  the  educational  program" 
during  implementation  of  the  remedial  plan  for  a  school  that  has  been  designated  as 
underperforming  (producing  this  remedial  plan  is  the  district's  responsibility).  As  in  the  area  of 
accountability,  DOE's  efforts  at  targeted  assistance  have  been  criticized  both  for  being 
implemented  too  slowly  and  for  involving  too  few  schools.  DOE  faces  three  capacity-related 
difficulties  in  implementing  supports  for  underperforming  schools. 

First,  such  support  is  likely  to  require  large  quantities  of  funding  and  expertise,  which  are  in 
short  supply.  Five  new  positions  added  to  the  School  and  District  Improvement  Planning  and 
Assistance  to  Underperforming  Schools  area  within  DOE  will  help,  but  more  staff  is  probably 
still  necessary,  particularly  if  the  targeted  assistance  function  is  thought  of  broadly,  as  including 
prevention  of  problems.  Contracting  with  educators  in  the  field  to  provide  assistance  to  then- 
peers  is  a  promising  direction,  but  even  if  DOE  draws  upon  experts  from  outside  its  staff,  it  still 
needs  fimds  in  order  to  pay  them  for  their  services  and  to  pay  DOE  staff  who  oversee  the 
process. 

Second,  it  will  be  carrying  out  this  support  role  in  a  climate  of  local  resistance  to  much  of  MERA 
and  frustration  at  what  many  educators  see  as  an  overly  punitive  state  presence.  The  shift  of 
accountabiUty  to  the  EMAC  may  help  somewhat,  in  that  DOE  could  play  "good  cop"  to  the 
EMAC's  "bad  cop." 

Finally,  even  some  people  who  generally  beheve  DOE  does  good  work  are  unsure  of  what  an 
effective  system  of  targeted  assistance  would  look  like.  This  is  one  of  the  most  important  and 
most  difficult  components  of  standards-based  reform.  Kentucky's  Highly  Skilled  Educators 
program  provides  one  interesting  model,  which  is  described  in  Appendix  1 . 


4.  Supporting  Local  Education  Governance  and  Management 

In  addition  to  changing  the  state's  responsibility  for  public  education,  MERA  also  included 
provisions  that  altered  responsibilities  for  govemance  and  management  of  schools  at  the  local 
level.  Under  Section  53  of  MERA  (M.G.L.  Chapter  71,  Section  59C),  each  public  school  in  the 
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Commonwealth  must  have  a  School  Council  consisting  of  the  principal,  parents  of  current 
students,  teachers,  other  community  representatives,  and,  in  high  schools,  at  least  one  student. 
The  School  Council's  role  is  to  "meet  regularly  with  the  principal  of  the  school  and  shall  assist  in 
the  identification  of  the  educational  needs  of  the  students  attending  the  school,  in  the  review  of 
the  annual  school  budget,  and  in  the  formulation  of  a  school  improvement  plan." 

According  to  MERA,  the  school  improvement  plans  must  address  class  size,  professional 
development  for  school  staff,  "allocation  of  any  professional  development  fimds  in  the  annual 
school  budget,"  "enhancement  of  parental  involvement  in  the  life  of  the  school,"  safety, 
discipline,  "a  welcoming  school  environment  characterized  by  tolerance  and  respect  for  all 
groups,"  extracurricular  activities,  inclusion  of  students  with  disabilities  in  regular  education 
programs,  language  acquisition  for  speakers  of  languages  other  than  English,  and  other  subjects 
deemed  appropriate  by  the  principal.  School  Committees  may  also  devolve  additional  authority 
to  School  Councils  if  they  so  choose. 

MERA  also  affected  governance  and  management  at  the  district  level.  Section  35  of  MERA 
(M.G.L.  Chapter  71,  Section  37)  limits  the  powers  of  School  Committees  to  hiring  and  firing  the 
Superintendent  of  Schools,  reviewing  and  approving  the  district  public  education  budget,  and 
establishing  educational  goals  and  policies  for  the  schools  in  the  district,  consistent  with 
statewide  laws  and  policies.  Superintendents  in  tum  appoint  principals,  who  gained  the  authority 
to  hire  and  fire  staff  in  their  schools  (Sec.44/MGL  Ch.71  sec.  42)  but  lost  tenure  (MERA  Sec.  45, 
repealing  MGL  Ch.  71  sec.  42A)  and  the  right  to  bargain  collectively  (Sec.43/MGL  Ch.  71  sec. 
41). 

Although  these  changes  all  affect  local,  rather  than  state,  decision  making  processes,  state 
authorities  have  had  a  role  in  their  implementation.  DOE's  1993  implementation  plan  assigns 
the  task  of  promoting  school  coimcils  and  site-based  management  to  the  Center  for  Iimovation,  a 
research  and  development  imit  within  DOE.  Although  implementation  of  school  councils  was  a 
major  priority  of  the  Center  for  Innovation,  it  was  less  so  for  DOE  as  a  whole.  The  major 
implementation  strategy  for  school  councils  was  the  issuing  of  guidelines  to  districts;  DOE  did 
not  provide  much  technical  assistance  for  districts  on  how  to  make  the  school  councils  work. 
The  Center  for  Innovation,  which  according  to  the  1993  Implementation  Plan  also  had  the 
responsibility  to  "provide  ongoing  information,  technical  assistance,  and  training  to  support 
school  districts  in  implementing  Education  Reform,"  was  disbanded  in  the  mid-1990s.  Its 
director  stayed  at  the  Department  in  another  position,  and  its  functions  were  distributed  among 
other  offices  within  DOE,  but  they  were  not  given  a  high  priority. 

In  its  audits  conducted  in  1999  and  2000,  the  Education  Management  Accountability  Board 
found  that  many  districts  it  studied  had  not  made  the  MERA  govemance  and  management 
changes  in  a  genuine  way.  The  districts  were  not  negotiating  individually  with  principals,  and 
School  Improvement  Plans  in  some  districts  did  not  address  "student  performance,  test  results,  or 
other  measurable  indicators  of  student  achievement." 

Although  people  outside  of  DOE  approved  of  some  of  the  changes  made  to  the  powers  of 
superintendents  and  school  committees,  they  also  said  that  the  school  councils  have  not  been 
especially  useful  and  that  DOE  could  have  done  a  better  job  of  supporting  them  after  they  were 
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first  created.  Some  interviewees  criticized  the  way  in  which  the  state  has  left  the  districts  on  their 
own  in  responding  to  the  new  standards  and  tests,  without  fimding  technical  assistance.  One 
wondered  whether  the  "loose-tight"  assumption  that  districts  would  be  able  to  choose  the  best 
way  to  meet  the  mandatory  goals  was  actually  warranted,  given  most  districts'  limited 
administrative  and  curriculum  planning  resources.  Some  have  noted  that  school  committees  and 
school  councils  are  unable  or  unwilling  to  provide  the  kind  of  support  and  assistance  that 
districts  need,  and  are  in  need  of  training  and  support  themselves.  Section  42  of  MERA  (MGL 
Ch.  71,  Sec.  38Q)  requires  districts'  professional  development  plans  to  include  training  for 
members  of  school  councils,  but  such  training  has  generally  not  been  widely  available. 

Building  local  districts'  own  intemal  capacity  for  reform  and  maintaining  public  support  for 
reform  efforts  are  also  key  to  the  state's  support  for  change  in  local  governance  and  management 
of  schools.  These  areas  have  been  problematic.  Survey  results  show  that  local  educators  believe 
the  state  has  unrealistic  expectations  of  districts  and  schools  under  Education  Reform.  Asked 
whether  they  strongly  agreed,  agreed,  disagreed,  or  strongly  disagreed  with  the  statement  "the 
state  has  reasonable  expectations  regarding  the  role  of  districts  in  implementing  the  Education 
Reform  Act,"  54.3%  disagreed  and  12.0%  strongly  disagreed.  Given  an  identical  statement 
about  schools,  51.0%  disagreed  and  10.1%  strongly  disagreed.  In  some  districts,  public  support 
for  MERA  (and  especially  MCAS)  is  weak. 

Lack  of  state-level  monitoring  and  support  for  local  governance  and  management  changes  is 
most  likely  the  result  of  both  scarce  resources  and  a  higher  priority  given  to  other  issues  such  as 
standard-setting  and  student  assessment.  As  in  other  Education  Reform  areas,  state  authorities 
backed  away  from  a  strong  regulatory  role  because  leaders  deemed  an  interventionist  state 
posture  to  be  inappropriate. 

Despite  the  state's  general  reluctance  to  intervene,  several  of  our  interviewees  said  that  technical 
assistance  and  training  for  School  Committees  and  School  Councils  would  be  helpful.  The  lines 
of  accountability  in  MERA  essentially  bypass  school  committees,  because  the  state  evaluates  the 
academic  performance  of  schools  rather  than  districts.  Formally,  principals  report  to 
superintendents,  who  are  hired  and  evaluated  by  school  committees,  but  they  also  must  be 
concerned  about  meeting  state-mandated  improvement  goals.  In  some  communities  where  the 
public  and  members  of  the  school  committee  are  opposed  to  MCAS  and  its  use  in  the  state 
accoimtability  system,  it  is  hard  for  educators  to  satisfy  both  state  and  local  authorities. 
Professional  development  or  training  for  School  Committee  members  might  ameliorate  this 
problem 


5.  Making  Coherent  State-Level  Policy 

MERA  posed  a  major  challenge  for  ongoing  formation  and  implementation  of  state  education 
policy.  It  required  not  only  that  the  structures  and  processes  of  the  K-12  public  education  system 
change,  but  also  that  they  change  in  a  systematic  and  coherent  way  consistent  with  standards- 
based  reform.  In  order  for  state  authorities  to  play  their  role  in  coordinating  local  efforts,  they 
must  themselves  act  in  a  coordinated  way  to  guide  and  monitor  education  activities. 
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Implementation  of  MERA  entailed  an  ongoing  policy  making  process  that  involved  the  Board  of 
Education,  the  Department  of  Education,  the  advisory  committees  to  the  Board  (see  MERA  , 
Section  3/  MGL  Ch.  15,  Section  IG,  MERA  Section  14/  MGL  Ch.  15 A,  Section  2),  the  Secretary 
of  Education,  the  legislature,  and  the  executive  branch,  in  particular  the  Governor.  As  discussed 
above  in  Part  II,  this  process  has  often  lacked  coordination,  since  different  actors  at  the  state 
level  have  different  sets  of  priorities  that  do  not  always  coincide. 

The  state-level  institutions  themselves  have  changed.  The  Board  of  Education  was  reduced  from 
17  to  9  members.  The  office  of  Secretary  of  Education,  which  under  Education  Reform  had 
responsibility  for  analyzing  "the  present  and  fixture  goals,  needs  and  requirements  of  public 
education  in  the  commonwealth"  and  recommending  "to  the  board  of  education  and  the  higher 
education  coordinating  council  comprehensive  goals  necessary  to  achieve  a  well-coordinated 
system  of  high  achievement  in  public  education  in  the  commonwealth,"  as  well  as  preparing  "an 
annual  master  plan  for  public  education"  and  reporting  to  the  public  each  year  on  the  condition 
of  public  education  in  the  Commonwealth,  was  eliminated  in  1996.  As  mentioned  earlier,  the 
Center  for  Innovation  within  DOE,  which  had  the  job  of  staffmg  the  advisory  committees,  no 
longer  exists. 

Listening  to  and  working  with  the  field  is  important  if  a  reform  of  the  scope  of  MERA  is  to 
succeed,  and  many  believe  that  state  authorities  have  not  engaged  enough  with  the  field.  The 
advisory  committees  and  public  reports  could  have  been  central  to  this  communication,  but  in 
many  people's  minds  have  not  achieved  their  potential. 

DOE  and  other  state-level  entities  have  begun  addressing  some  of  the  problems  of  policy 
coordination  and  communication  with  the  field.  It  appears  that  the  uncertainty  over  the 
accountability  roles  of  DOE  and  EMAC  is  in  the  process  of  resolution.   DOE's  Associate 
Commissioner  for  Assessment  has  made  several  well-received  visits  to  school  districts  to  discuss 
MCAS  with  local  educators.  A  few  of  our  interviewees  said  that  the  current  leadership  of  BOE 
and  DOE  has  been  more  willing  to  reach  out  and  Usten  to  the  field  than  they  were  during  John 
Silber's  term  as  Board  Chair.  In  December,  2000,  the  DOE  added  a  Director  of  Community 
Outreach  who  serves  as  a  liaison  between  DOE  and  the  field.  Improved  commimication  may 
restore  cooperation  between  state  and  local  education  authorities. 


6.  Collecting  and  Using  Information  to  Improve  the  Performance  of  the 
Educational  System 

Coordination  of  policy  making,  accountability,  and  communication  requires  that  the  state  collect 
appropriate  information  and  use  it  in  ways  that  contribute  to  system  improvement.  This  is  true 
not  only  of  the  state  education  system,  but  also  of  any  other  system  undergoing  reform  or 
continuous  improvement  (Kaplan  and  Norton,  1992). 

MERA  contains  several  specific  provisions  related  to  the  collection  and  use  of  data.  One  of  the 
most  important  of  these  is  the  mandate  for  collection  of  data  on  school  and  student  performance 
and  the  use  of  that  data  to  hold  schools  accountable  (se<^  above).  Others  include  a  requirement 
(MERA  Section  29;  MGL  Ch  69,  Sec.  IB)  that  the  board  publish  profiles  of  each  public  school 
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and  district,  including  information  on  "student  achievement  of  performance  goals,  school 
spending,  special  programs,  curriculum  offerings,  qualifications  of  teaching  staff,  and  other 
information  which  may  be  pertinent  to  teachers,  parents,  students,  and  elected  officials  regarding 
the  performance  of  said  schools  and  school  districts."  The  1993  Implementation  Plan  assigned 
this  function  to  the  Executive  Office  of  Educational  Affairs.    The  same  section  of  MERA  also 
directs  the  Board  to  identify  schools  and  districts  that  are  especially  successful  in  improving 
student  performance  and  "undertake  to  analyze  and  publish  the  strategies  employed  by  such 
schools  and  districts  for  the  purpose  of  recognizing  the  efforts  of  the  educators  involved  and  of 
encouraging  the  replication,  where  appropriate,  of  their  successful  strategies."  According  to  the 
Implementation  Plan,  this  function  was  to  be  carried  out  by  DOE's  division  of  Evaluation, 
Planning,  and  Research  in  tandem  with  EOEA. 

The  requirement  that  the  Secretary  of  Education  produce  an  aimual  report  on  the  condition  of 
education  in  the  Commonwealth  (MERA  Section  16,  amending  MGL  Ch.lSA,  Sec.3)  also  is  part 
of  the  overall  system  for  collecting  and  using  information.  This  section  of  the  law  was  repealed 
in  1996  when  the  office  of  Secretary  of  Education  was  eliminated.  The  section  of  MERA  on 
school  choice  (see  below)  requires  a  parent  information  system  for  choice,  including  but  not 
limited  to  "information  on  special  programs  offered  by  the  school,  philosophy  of  the  school, 
number  of  spaces  available,  transportation  plans,  class  sizes,  teacher/student  ratios,  and  data  and 
information  on  school  performance  that  indicate  its  quality."  The  law  stipulates  that  this 
information  include  the  school  profiles  (MERA  sec.  61 /MGL  ch.  76,  sec.l2B  (h)).  Like  the 
dissemination  of  successfiil  models  of  education,  this  fimction  was  to  be  shared  between  the 
Evaluation,  Planning,  and  Research  division  of  DOE  and  the  EOEA. 

In  addition.  Section  79  of  MERA  directs  the  Governor  to  appoint  an  Education  Reform  Review 
Commission  with  authority  to  "monitor  the  extent  to  which  the  commonwealth  has  carried  out  its 
responsibilities  under  this  act  and  the  extent  to  which  such  efforts  have  brought  about 
educational  reform  in  the  commonwealth."  The  law  gives  the  Commission  the  "right  to  request 
from  the  board  of  education,  the  secretary  of  education,  and  the  joint  committee  on  education, 
arts,  and  humanities  information  and  data  pertinent"  to  its  charge,  including  the  information 
being  used  to  assess  school  and  student  performance.  The  Education  Reform  Review 
Commission  also  was  intended  to  have  data-gathering  capacity,  but  has  not  because  of  the  delay 
in  funding  it  at  all  and  the  low  levels  of  funding  it  ultimately  received. 

Finally,  the  legislature's  Joint  Committee  on  Education,  Arts,  and  Humanities  could  conceivably 
have  played  a  major  role  in  oversight  of  MERA  implementation,  but  it  has  not.  The  Committee 
has  a  very  small  staff. 

In  addition  to  the  data-gathering  functions  explicitly  outlined  in  MERA,  several  of  the  law's 
provisions,  such  as  accountability  and  teacher  quality,  implicitly  require  data  analysis. 
Collection  and  use  of  information  about  education  by  the  state  includes  five  subtasks.  First,  the 
state  needs  to  have  a  strategy  for  identifying  and  collecting  appropriate  data  that  meet  the  needs 
of  state  and  local  data  users.  Second,  data-gathering  instruments  must  be  designed  and  data 
collected,  either  from  local  education  authorities  or  directly  by  the  state.  Third,  the  accuracy  of 
data  collected  from  local  authorities  must  be  verified.  Fourth,  the  data  must  be  analyzed  and 
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made  available  in  a  form  that  is  useful  to  decision  makers.  Fifth,  the  data  must  actually  be  used 
in  evaluation  or  decision  processes. 

From  the  beginning,  the  state's  capacity  to  collect  and  use  data  has  been  an  area  of  concern. 
In  1994,  Rep.  Mark  Roosevelt  stated  that  DOE  needed  to  improve  its  data  collection  capacity 
because  there  was  not  enough  accurate,  understandable  data  about  school  spending  (p.  5).  The 
MBAE's  1995  report  Within  Our  Reach  also  calls  for  more  data  collection,  as  does  the  EMAB 
report  issued  in  2000  (p.  40). 

One  of  the  entities  that  mitially  had  the  most  responsibility  for  data  collection,  the  Executive 
Office  for  Educational  Affairs,  no  longer  exists  at  all.  The  other,  DOE's  Division  of  Evaluation, 
Plarming,  and  Research,  first  became  part  of  the  Division  of  Finance  and  Accountability  and  by 
2001  did  not  appear  at  all  on  the  organizational  chart.  One  Evaluation  and  Research  Specialist 
reported  directly  to  the  Deputy  Commissioner  for  Academic  Affairs,  and  there  were  Data 
Collection,  Data  Processing,  and  Data  Analysis  and  Reporting  units  within  the  Accountability 
and  Targeted  Assistance  operation.  The  Education  Reform  Review  Commission  did  not  have 
any  staff  or  fimding  until  1997,  and  still  is  funded  quite  modestly  relative  to  the  size  of  its 
statutory  task.  Most  of  our  interviewees  cited  the  lack  of  accurate  data  and  policy  evaluation 
based  upon  it  as  a  major  shortcoming  of  Education  Reform  implementation.  For  example,  the 
state  is  attempting  to  enhance  the  quality  and  preparation  of  its  teaching  force  without 
comprehensive  data  on  how  and  where  current  Massachusetts  teachers  were  prepared  and  which 
areas  are  likeliest  to  experience  teacher  shortages  in  the  near  future.  In  another  areas,  an 
interview  said,  "My  impression  as  a  completely  uninformed  outsider  is  that  there 's  been  much 
more  interest  in  using  technology  for  flashy  applications,  like  the... Virtual  Education  Space 
instead  of  the  sort  of  bread  and  butter  stuff  of  you  know,  where  do  kids  go  and  what  classes  did 
they  take,  and  how  did  they  do.  " 

Data  collection  and  use  is  a  crucial  part  of  the  state-local  relationship.  Our  survey  indicates 
moderate  levels  of  local  dissatisfaction  with  the  state's  collection  and  use  of  data,  and  an  almost 
even  spht  (slightly  favoring  "agree")  on  whether  data  from  DOE  are  helpful  in  improving  the 
quality  of  education: 


Table  8  -  Educators'  Experience  with  the  State's  Use  of  Data 
("Don't  know"  answers  omitted.) 


Type  of  Experience 

Strongly 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

The  Department  of  Education  collects  appropriate 
data  from  schools 

11.2% 

38.5% 

38.1% 

3.6% 

The  Department  of  Education  uses  these  data 
appropriately 

16.1% 

39.4% 

20.4% 

1.8% 

The  data  my  district  gets  from  the  Department  of 
Education  are  helpfial  in  our  efforts  to  improve  the 
quality  of  education 

11.5% 

34.6% 

38.0% 

3.6% 
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The  first  two  items  were  not  asked  of  teachers,  but  the  third  item,  measuring  how  helpful  data 
from  Department  of  Education  are  for  local  efforts  to  improve  the  quahty  of  education,  is  viewed 
much  less  favorably  by  teachers  than  by  administrators.  The  local  educators  we  interviewed  said 
that  the  policy  or  educational  context  for  the  state's  data  reports  and  requests  generally  is  not 
well-explained,  which  makes  the  requests  and  reports  seem  incoherent. 

DOE  leaders  are  aware  of  the  importance  of  solving  problems  in  the  area  of  data  collection  and 
use.  At  the  April  24,  2001  Board  of  Education  meeting.  Commissioner  David  Driscoll  observed 
that  local  educators  will  not  pay  close  attention  to  the  quality  of  the  data  they  are  submitting  to 
the  state  unless  they  believe  that  the  resulting  product  that  the  state  sends  back  to  them  will  be 
useful. 

Many  people  in  the  state  are  optimistic  about  the  DOE's  Data  Mart,  which  will  be  on  line  soon. 
In  Spring,  2001,  the  Educator  Quality  group  began  gathering  information  on  mentoring  and 
induction  programs  for  teachers.  In  the  summer  and  fall,  data  collection  on  educational  leaders 
will  occur  as  part  of  a  larger  project  on  developing  educational  leaders.  Leaders  in  the  Board, 
the  Department,  and  the  field  support  building  more  data  collection  and  evaluation  capacity  at 
DOE.  As  one  interviewee  put  it,  "I  think  it  would  be  useful  to  convene  DOE  data  people,  district 
data  people  (from  big  districts  and  small  ones — they  have  different  issues),  PALMS  data 
implementation  advisors,  and  maybe  some  data  experts  to  figure  out  what  the  data  management 
system  of  the  state  should  be — to  help  DOE  get  the  data  it  needs,  districts  deal  with  data  requests 
efficiently,  and  district  leaders  to  figure  out  which  kids  are  learning  and  which  programs  are 
working,  or  not.  " 

Capacity  to  collect  and  analyze  information  on  education  policy  need  not  be  built  only  within 
state  agencies.  In  Chicago,  the  privately  funded  Consortium  on  Chicago  School  Research 
organization  performs  evaluation  and  data  collection  related  to  the  1988  education  reform  law. 
More  information  on  the  Consortium  is  included  in  Appendix  1 . 

Auditing,  both  of  district  and  school  MERA  implementation  and  of  the  accuracy  of  the  data  they 
send  to  DOE,  remains  an  issue.    The  State  Auditor's  1997  report  on  Education  Reform 
implementation  found  little  monitoring  of  whether  schools  were  actually  implementing  required 
changes  like  increased  structured  learning  time,  lack  of  fiscal  controls,  lack  of  data  collection, 
lack  of  quality  assurance,  ineffective  monitoring,  and  inadequate  guidelines.  EMAB  also  found 
that  many  districts  fail  to  follow  the  protocol  for  reporting  spending  of  state  funds  in  the  areas 
such  as  professional  development  in  which  they  in  effect  receive  categorical  grants  (i.e.  money 
that  is  designated  to  be  spent  for  a  specific  purpose). 


7.  Enhancing  Educator  Quality  and  Education  as  a  Profession 

We  address  issues  related  to  certification,  recertification,  and  professional  development  for 
educators  together  in  this  section  because  they  are  closely  related  in  practice  and  policy.  Under 
MERA,  educators  no  longer  have  lifetime  certification,  and  in  order  to  be  recertified  at  five-year 
intervals  they  must  demonstrate  that  they  have  participated  in  professional  development.  The 
DOE's  Office  of  Educator  Quality  addresses  both  certification  and  professional  development 
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ftmctions  for  the  Department,  and  has  also  implemented  state  programs  to  provide  incentives  for 
new  entrants  to  the  teaching  profession  and  to  enable  alternate  routes  to  educator  certification. 

The  DOE  role  in  enhancing  the  teaching  profession  is  fraught  with  tension.  DOE  both  regulates 
entry  into  the  profession  and  provides  funding  and  services  for  professional  development.  The 
elimination  of  regional  DOE  centers  in  the  1980s  seemed  to  take  the  department  out  of  the 
professional  development  business,  but  the  increased  professional  development  requirements  for 
recertification  represents  an  increased  regulatory  presence.  Although  the  state  has  the 
responsibility  for  implementing  MERA,  most  funds  for  professional  development  go  directly  to 
local  districts  rather  than  into  state  programs.  This  funding  pattern  has  made  it  hard  for  DOE  to 
lead  in  professional  development.  At  the  same  time,  some  believe  that  DOE  played  a  crucial 
leadership  role  in  legislation  passed  following  the  1998  teacher  test  controversy.  The 
Department's  involvement  in  the  "12  to  62"  initiative  could  serve  as  a  model  for  how  DOE 
might  be  more  broadly  influential  in  the  legislative  process. 

Educator  Certification  and  Recertification 

Section  41  of  MERA  (M.G.L.  Ch.  71,  Sec.  38G)  gives  the  Board  of  Education  authority  to  grant 
provisional  and  standard  certificates  of  five  years'  duration  and  expands  the  range  of  routes  to 
certification.  All  teachers  are  to  pass  competency  tests  before  earning  certification.  The  same 
section  of  the  law  directs  the  Department  of  Education  to  "coordinate  the  training  efforts  of 
districts,"  "provide  orientation  programs  for  support  team  members,"  and  "devise  standardized 
criteria  and  forms  for  a  final  comprehensive  evaluation  of  each  provisional  teacher."  According 
to  the  law,  certification  fees  "shall  be  established  so  as  to  allow  the  department's  bureau  of 
teacher  certification  to  operate  at  no  cost  to  the  commonwealth."  However,  the  revenues  firom 
certification  fees  do  not  go  to  the  Department  of  Education,  but  rather  into  the  Commonwealth's 
general  fimd. 

Because  of  the  changed  roles  of  principals  under  Education  Reform,  Section  90  of  MERA  directs 
the  Board  of  Education  to  "(i)  establish  new  standards  for  the  certification  of  principals  in  light 
of  the  responsibilities  given  to  principals  under  this  act,  and  (ii)  fashion  procedures  for 
recertifying  persons  serving  as  principals  under  these  new  standards."  Both  for  teachers  and 
principals,  certification  programs  are  to  be  consistent  with  MERA  and  the  Curriculum 
Frameworks. 

MERA  also  has  provisions  designed  to  change  the  membership  of  the  Commonwealth's  teacher 
workforce.  Section  22  (MGL  Ch.lSA  sec.  19A)  created  the  Attracting  Excellence  to  Teaching 
program,  which  provides  assistance  with  repayment  of  educational  loans  for  students  in  the  top 
quarter  of  their  college  graduating  classes  who  enter  teaching  after  July  1,  1994.  Section  83  puts 
in  place  early  retirement  incentives  for  current  school  personnel.  The  Executive  Office  for 
Educational  Affairs  was  given  responsibility  for  administering  the  Attracting  Excellence  to 
Teaching  program  in  the  1993  implementation  plan,  with  the  other  certification  and  personnel- 
related  fimctions  divided  among  the  Legal  Counsel,  Professional  Standards  Development 
Services,  and  Certification  and  Credentialling  Services  areas  of  DOE. 
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Assessments  of  the  MERA  provisions  in  this  area  and  of  DOE's  capacity  to  implement  them  are 
mixed.  Some  applaud  the  alignment  of  educator  preparation  with  the  curriculum  frameworks. 
Others  criticize  the  changes  in  certification  and  recertification  as  confusing,  and  claim  that  DOE 
has  not  done  a  good  job  of  administering  them.  In  particular,  according  to  some  the  first 
recertification  of  teachers  was  done  without  "quality  control"  despite  the  intent  of  the  law.  The 
process  of  writing  the  new  regulations  also  struck  some  respondents  as  unnecessarily  conflictual. 
When  asked  to  evaluate  the  extent  to  which  the  state  provided  support  for  "developing  human 
resources  policies  to  improve  staff  quality,"  43.0%  of  survey  respondents  said  the  state's  support 
in  that  area  had  been  "poor."  Only  1 1.3%  of  respondents  rated  the  state's  support  as  "good"  or 
"excellent,"  and  30.9%  chose  "fair." 

Education  Reform  puts  principals  in  an  especially  difficult  position.  Changes  in  the  process  for 
dismissing  principals  have  left  them  with  less  job  protection  than  most  teachers,  and  some 
believe  that  this  impedes  reform.  There  has  been  a  lot  of  mobility  among  principals,  and  less 
ability  to  take  risks.  At  the  same  time,  EMAB's  audit  findings  suggest  that  principals  are  no 
more  able  to  dismiss  underperforming  teachers  than  they  were  before  MERA. 

Professional  Development 

Professional  development  is  critically  important  for  education  reform.  According  to  one  study  of 
reform  implementation  in  12  schools  in  3  states,  professional  development  for  teachers  is  the  key 
to  making  policy  reforms  actually  affect  educational  practice  (Goertz,  Floden,  and  O'Day,  1995). 
In  addition  to  being  crucial,  state-led  professional  development  is  also  quite  difficult.  As  Lusi 
says  in  her  study  of  reform  in  Kentucky  and  Vermont,  "Not  only  is  the  state  trying  to  change  the 
practice  of  a  large  number  of  practitioners  over  whom  it  has  little  control  and  no  proximity;  in 
addition,  it  is  trying  to  make  this  change  in  a  profession  where  good  practice  is  nearly  impossible 
to  clearly  specify  and  in  an  environment  in  which  it  is  difficult  to  predict  the  effect  of  its  actions" 
(Lusi,  1997,  p.  11). 

The  approach  taken  thus  far  to  professional  development  in  MERA  exemplifies  the  "tight-loose" 
assumption  about  the  state's  role  in  reform.  Local  districts  received  state  funds  for  professional 
development  and  were  expected  to  use  it  in  ways  that  supported  reform.  In  practice,  many 
districts  have  not  known  how  to  use  the  fiinds  effectively,  and  the  state  has  had  insufficient 
resources  to  use  in  guiding  them.  The  Department  has  begun  building  capacity  in  a  reorganized 
"Educator  Quality"  cluster,  and  several  interviewees  spoke  very  highly  of  the  work  this  team  has 
begun. 

The  MERA  provisions  that  concern  professional  development  are  related  to  the  evaluation  and 
certification  of  educational  personnel.  One  of  the  criteria  according  to  which  superintendents  are 
to  evaluate  the  performance  of  teachers,  principals,  and  administrators  (MERA  sec.40/MGL 
Ch.71  sec.  38)  is  successful  implementation  by  each  individual  of  a  professional  development 
plan.  Section  41  of  MERA  (M.G.L.  Ch.71  sec.38G)  directs  the  Board  of  Education  to  estabHsh 
policies  and  guidelines  on  professional  development  requirements  for  maintaining  skills  and 
subject  matter  knowledge  in  particular  certificate  areas  and  says  that  applicants  for  renewal  of 
educator  certification  "shall  submit  documentation  demonstrating  their  fulfillment  of  the 
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professional  development  requirements  established  by  the  board  for  the  certificates  they  seek  to 


renew." 


Not  only  individuals,  but  also  districts,  must  implement  professional  development  plans  (MERA 
Sec.  42/MGL  Ch.71  sec.38Q).  The  plans  are  required  to  "include  training  in  the  teaching  of  new 
curriculum  frameworks  and  other  skills  required  for  the  effective  implementation  of  this  act, 
including  participatory  decision  making,  and  parent  and  community  involvement."  Finally,  the 
"commissioner  of  education  for  the  commonwealth  shall  prepare  each  year  a  plan  for  providing 
statewide  assistance  in  the  preparation  and  implementation  of  professional  development  plans," 
in  consultation  with  the  secretary  of  education  and  the  Higher  Education  Coordinating  Council. 
To  facilitate  MERA  implementation,  local  professional  development  activities  ought  to  be 
aligned  with  Education  Reform,  but  the  provision  of  professional  development  fimds  to  districts 
on  a  per-student  basis  complicates  DOE's  ability  to  produce  such  alignment. 

The  1993  implementation  plan  assigned  the  task  of  establishing  "guidelines  and  a  statewide  plan 
for  professional  development  and  recertification"  to  the  Instructional  and  Curriculum  Services 
area  of  DOE. 

The  draft  of  the  Department  of  Education's  2001  statewide  professional  development  plan 
identifies  as  priorities  expanding  teachers'  knowledge  of  the  subject  matter  of  the  school 
curriculum,  extending  teachers'  familiarity  with  and  use  of  Massachusetts  learning  standards  in 
planning  curricula;  helping  teachers  understand  the  strengths  and  limitations  of  a  range  of 
commonly  used  instructional  programs  and  pedagogical  practices  in  their  disciplines,  and  raising 
educator,  parent,  and  community  expectations  for  student  achievement.  These  objectives  relate 
both  to  general  enhancement  of  teachers'  knowledge  and  skill  and  to  specific  improvement  in 
their  knowledge  specific  to  Education  Reform  and  other  state  initiatives. 

Many  in  the  state  and  in  the  field  either  do  not  believe  that  the  state  should  itself  provide 
professional  development  or  do  not  believe  that  it  can  provide  professional  development.  There 
is  more  willingness  to  accept  a  state  role  in  the  kinds  of  professional  development  that  relate 
specifically  to  MERA.  Interviewees  also  often  recommended  that  DOE  play  a  brokering  role,  to 
develop  and  manage  a  network  of  organizations  and  consultants  who  would  provide  assistance  to 
districts  and  schools.  People  who  have  been  involved  in  PALMS  see  its  professional 
development  components  as  a  model  for  this  sort  of  network.  As  one  interviewee  put  it: 

The  Department  should  hire  someone,  or  contract  with  someone,  to  spend  a  year  finding  the 
best  professional  development  going  on  right  now,  and  set  it  up  so  that  those  people  are 
available  to  district  that  are  ready  to  use  them.  In  some  other  states,  there  are  all  kinds  of 
resources  available  to  help  underperforming  districts — networks  of  universities,  colleges, 
independent  professional  development  providers.  That  hasn  't  happened  in  Massachusetts — 
it's  a  huge  loss.  I'm  not  sure  the  Department  should  provide  the  professional  development, 
but  they  should  set  up  the  systems  that  make  it  possible  to  do  it — bring  together  networks  of 
higher  ed,  set  up  teacher  centers,  opportunities  for  sharing  of  practice. 

Respondents  to  our  survey  were  critical  of  the  state's  support  for  professional  development.  In 
the  area  of  professional  development  for  teachers,  34.7%  of  respondents  rated  the  state's  support 
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for  local  efforts  as  "good"  or  "excellent."  61.2%  rated  it  as  "fair"  or  "poor,"  and  the  remainder 
of  respondents  selected  "don't  know."  In  the  area  of  professional  development  for 
administrators,  21.7%  chose  "good"  or  "excellent,"  64.4%  chose  "poor"  or  "fair,"  and  the 
remainder  chose  "don't  know."  They  also  expressed  concern  in  an  open-ended  question  about 
issues  such  as  a  perceived  focus  on  "punishing  teachers  rather  than  on  improving  teachers," 
failure  of  professional  development  to  keep  pace  with  changing  state  requirements,  and  lack  of 
time  for  professional  development  given  all  the  other  state  initiatives  to  which  educators  must 
respond. 

As  in  other  areas,  the  state  has  been  criticized  for  insufficient  oversight  of  districts'  and 
individuals'  professional  development  plans  and  districts'  use  of  Education  Reform  money  for 
professional  development,  which  is  one  of  the  four  areas  in  which  the  state  sets  a  recommended 
per-pupil  expenditure  in  the  foundation  budget.  According  to  the  state,  districts  should  spend  a 
set  amount  of  money  per  student  ($125  in  FY  2000)  per  enrolled  student  on  professional 
development,  and  if  they  do  not  they  must  send  an  explanation  in  writing  to  the  Commissioner  of 
Education.  Some  of  the  districts  audited  by  EMAB  were  not  spending  the  required  amount  of 
money  and  were  also  not  sending  the  required  explanation  to  the  Commissioner.  According  to 
the  Massachusetts  Teachers'  Association's  2000  study  of  implementation  of  the  mathematics 
curriculum  framework,  only  54%  of  districts  spent  the  required  amount  on  professional 
development  in  FY  1999,  and  only  28%  had  budgeted  to  do  so  in  FY  2000.  The  MTA  study  also 
criticized  the  DOE  for  failing  to  produce  statewide  professional  development  plans  and  for 
carrying  out  insufficient  informational  sessions  for  districts  on  the  preparation  of  district- wide 
professional  development  plans.  According  to  the  MTA,  only  6.6%  of  districts  were  able  to 
participate  in  such  sessions.  The  lack  of  access  to  the  sessions  results  firom  the  low  level  of 
funding  for  state  professional  development  activities. 

The  basic  problem  is  that,  in  addition  to  uncertainty  about  its  appropriate  role  in  professional 
development,  DOE  lacks  sufficient  staff  to  monitor  and  guide  district  efforts.  Interviewees 
criticized  the  state  for  requiring  that  teachers  amass  a  set  number  of  professional  development 
points  in  order  to  be  recertified  and  then  not  being  able  to  enforce  the  requirement  through 
anything  more  than  "casual  spot-checking."  The  state  is  also  unable  to  audit  providers  to  ensure 
that  the  courses  offered  are  appropriate  for  recertification. 

Professional  development  is  a  key  area  for  capacity  building  both  for  practical  reasons  (its  link  to 
the  quality  of  teaching  and  to  the  implementation  of  standards-based  reform)  and  because  of  the 
message  that  an  increased  state  commitment  to  professional  development  would  send.  Right 
now,  according  to  one  of  our  interviewees,  the  state's  decision  to  enforce  accountability  strictly 
while  being  lenient  about  professional  development  regulations  sends  a  message  that  teacher 
education  is  less  relevant  to  education  reform  than  is  accountability.  Increased  capacity  need  not 
be  more  staff  in  DOE  providing  professional  development — in  fact,  it  is  probably  preferable  not 
to  go  this  direction.  Instead,  the  state  could  build  a  network  of  "preferred  providers"  in  which  it 
would  serve  as  a  monitor  and  financial  supporter  of  others'  initiatives. 
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8.  Ensuring  Readiness  to  Learn  Through  Early  Childhood  Education 
Programs 

Section  70  of  MERA  creates  a  special  commission  "to  develop  a  plan  to  provide  pre- 
kindergarten  children  ages  three  to  four  the  opportunity  to  participate  in  a  developmentally 
appropriate  early  childhood  education  program,"  including  the  possibility  of  charging  tuition  on 
a  sliding  scale,  contracting  with  private  providers  and  Head  Start  programs,  outreach  to  parents, 
and  full-day  kindergartens.  According  to  the  1993  implementation  plan,  this  commission  was 
staffed  by  DOE's  Early  Learning  Services  office,  which  also  established  a  demonstration  project 
on  parent  outreach  and  education.  Learning  Support  Services  was  to  prepare  a  plan  of 
comprehensive  child  and  family  services. 

Expansion  of  early  childhood  education  in  the  Commonwealth  predates  MERA.  The  School 
Improvement  Act  of  1985  included  the  Chapter  188  early  childhood  program,  which  was  revised 
and  renamed  Community  Partnerships  for  Children  in  FY  1993.  In  FY  1996,  Community 
Partnerships  for  Children  became  the  preschool  component  of  MERA.  Early  childhood 
education  was  a  major  priority  of  former  Board  of  Education  Chair  John  Silber,  who  took  office 
in  1996.  According  to  the  State  Auditor's  report  on  Community  Partnerships,  program  funding 
grew  247%  from  1996  to  1999.  Despite  the  increased  state  financial  commitment,  some  local 
respondents  to  our  survey  mentioned  early  childhood  education  as  an  area  of  Education  Reform 
that  has  not  yet  been  adequately  implemented. 

One  possibility  is  that  despite  the  increased  investment  in  Community  Partnerships  for  Children, 
there  is  a  mismatch  between  community  needs  and  available  resources.  Children  participating  in 
Community  Partnerships  for  Children  programs  must  be  preschool-aged  children  of  working 
parents.  The  State  Auditor's  report  cites  numerous  instances  in  which  money  intended  to  be 
used  in  CPC  was  instead  used  to  fund  care  for  infants,  toddlers,  at-risk  children  of  varying  ages, 
and  children  whose  parents  were  unemployed.  These  kinds  of  care  are  all  as  important  as  care 
for  preschoolers  whose  parents  work,  and  all  could  contribute  to  children's  school  readiness. 

Our  interviewees'  perceptions  of  DOE's  spending  on  early  care  and  education  were  quite  mixed. 
Many  of  the  people  we  spoke  with  praised  CPC  as  a  flexible  program  that  gets  money  to  where  it 
is  needed.  However,  others  saw  chaos  rather  than  flexibility,  which  is  consistent  with  the 
Auditor's  report.  The  state  early  childhood  education  program  is  a  prime  example  of  an  area  in 
which  funding  for  programs  has  far  outstripped  resources  available  for  administration  and 
evaluation  to  determine  whether  the  program  is  being  implemented  correctly  and  effectively. 
Early  Learning  Services  has  only  22  staff  members.  According  to  the  audit,  intemal  controls 
over  the  use  of  CPC  money  were  inadequate,  resulting  in  state  fimds  being  spent  in  inappropriate 
ways  or  on  children  not  ehgible  for  CPC  services.  Many  of  the  Community  Partnership  Councils 
did  not  fimction  in  the  collaborative  way  envisioned  in  the  law,  with  many  members  not 
attending  meetings  and  many  meetings  not  documented.  CPC's  that  did  not  comply  with 
reporting  requirements  were  not  sanctioned. 

The  Auditor  notes  that  DOE  has  undertaken  improvements  in  the  control  over  this  program. 
Recently,  several  staff  members  with  research  responsibilities  joined  the  Early  Childhood  cluster 
of  DOE. 
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9.  Implementing  Choice  and  Charter  Schools 

School  choice  predates  Education  Reform.  However,  MERA  changed  state  law  on  school  choice 
by  requiring  districts  either  to  participate  or  to  take  a  public  vote  not  to,  and  also  changed  the 
financial  implications  of  inter-district  choice. 

Section  61  of  MERA  (M.G.L.  Ch.  76,  Section  12B)  states  that  children  may  attend  school  in  a 
district  in  which  they  do  not  reside,  provided  that  the  district  where  they  wish  to  attend  has  space 
available  and  has  not  filed  a  resolution  withdrawing  from  the  Choice  Program  (voted  on  by  the 
School  Committee)  with  DOE.  The  student's  sending  district  loses  state  aid  for  that  student, 
unless  the  district  is  spending  below  the  foundation  level.  MERA  also  directs  the  Board  of 
Education  to  establish  and  maintain  a  parent  information  system  to  facilitate  the  use  of  school 
choice.  In  the  1993  Implementation  Plan,  DOE's  Information  and  Outreach  services  team  was 
given  responsibility  for  administering  school  choice  participation  and  reimbursement.  The 
Executive  Office  of  Educational  Affairs  and  DOE's  Evaluation,  Planning,  and  Research  office 
shared  responsibility  for  the  parent  information  system. 

MERA  also  expanded  the  range  of  available  public  school  choices  by  providing  for  charter 
schools  in  Section  55  (M.G.L.  Ch.  71,  Sec.  89).  Initially,  the  DOE  did  not  run  the  charter  school 
program.  Instead,  it  was  one  of  the  responsibilities  of  the  Executive  Office  for  Educational 
Affairs.  Even  after  the  EOEA  was  eliminated  and  charter  schools  transferred  to  DOE,  the 
charter  school  office  remained  in  Boston  rather  than  in  Maiden  with  the  rest  of  the  department. 

Including  choice  and  charters  in  MERA  has  affected  some  local  educators'  perceptions  of  the 
overall  reform  effort.  When  students  leave  their  home  district's  public  schools  to  attend  either 
another  district  or  a  charter  school,  the  home  district  loses  money,  a  difficult  consequence  for 
some  smaller  districts  to  bear.  Choice  and  charters  thus  become,  to  many  educators,  one  more 
piece  of  evidence  that  the  state  is  "out  to  get"  the  public  schools. 

The  charter  school  office  has  effectively  carried  out  a  variety  of  functions  with  a  limited  staff. 
Staffing  of  the  charter  school  program  only  recently  increased  from  2  to  7  FTE  (it  also  had  one 
consultant  in  1995  and  one  in  1997),  has  a  wide  range  of  responsibilities.  It  evaluates  charter 
applications,  coordinates  aimual  visits  to  each  of  the  40  schools,  and  contracts  for  re-chartering 
visits  to  each  school  every  five  years.  The  annual  reviews  include  organizational  capacity, 
educational  programs,  and  an  independent  financial  audit.  On  an  ongoing  basis,  the  Charter 
School  office  works  to  ensure  that  charter  school  boards  of  trustees  are  following  state 
regulations  and  ethics  guidelines  and  that  they  are  negotiating  good  contracts  with  their 
Education  Management  Organization  if  they  work  with  one.  It  also  provides  assistance  with 
special  education  programs. 

The  charter  school  office  contracts  with  an  outside  organization  to  evaluate  the  performance  of 
the  schools  under  its  jurisdiction.  Many  in  the  state  view  the  resulting  accountability  system  as  a 
model  of  what  the  state  can  produce,  given  adequate  resources,  and  believe  that  the  evaluation  of 
all  schools  should  be  conducted  following  the  charter  school  model.  Charter  schools' 
evaluations  include  site  visits  and  consideration  of  evidence  of  student  learning  other  than  the 
MCAS. 


State  Capacity  to  Implement  the  Massachusetts  Education  Reform  Act  of  1 993  48 


10.  Funding  Public  Education 

Sections  32-34  of  MERA  fundamentally  changed  the  state's  role  in  finance  of  public  education 
(MGL  Chapter  70).  In  1993,  many  educators  and  political  observers  were  skeptical  that  the 
state's  elected  officials  would  actually  deliver  on  their  promise  to  spend  $1.3  billion  over  seven 
years  to  bring  every  school  district's  per-pupil  spending  up  to  the  foundation  level.  According  to 
some  of  our  interviewees,  this  skepticism  may  even  have  lessened  the  sense  of  urgency  educators 
felt  to  make  the  other  changes  mandated  by  MERA — they  believed  that  the  state  would  not 
follow  through,  based  on  the  precedents  set  by  earlier  state  school  reform  initiatives. 

Despite  these  predictions,  and  aided  by  an  expanding  economy,  the  state  fiilly  fimded  foundation 
aid  to  local  school  districts  under  MERA.  Although  some  of  our  interviewees  quibbled  with  the 
details  of  the  state's  funding  formula,  overall  they  were  impressed  that  the  state  has  delivered  on 
its  promise  to  fund  MERA  fiilly  and  expressed  the  belief  that  state  aid  to  education  should  be 
maintained  now  that  all  districts  have  been  brought  up  to  foundation  level. 

However,  the  local  educators  we  surveyed  still  singled  out  fimding  as  a  problematic  area  in  their 
responses  to  open-ended  questions.  Their  complaints  focused  more  on  adequacy  of  funding  than 
on  equity.  To  many  local  educators,  standards-based  reform  represents  a  state  mandate  that  the 
state  should  pay  for.  In  some  districts,  much  of  the  new  state  aid  went  to  meet  rising  special 
education  needs  (MASS,  1995). 

One  reason  for  the  persistence  of  funding  problems  despite  all  districts'  being  at  or  above  the 
foundation  level  Ues  in  the  determination  of  the  foundation  budget.  The  Foundation  Budget 
Review  Commission  was  intended  to  review  the  foimdation  level  annually  in  order  to  ensure  that 
it  accurately  reflected  the  cost  of  educating  students  to  meet  the  state  standards.  Because  former 
Govemor  Weld  did  not  reappoint  the  commission  after  its  first  members'  terms  expired,  the 
foundation  budget  has  not  been  subject  to  such  review.  The  foundation  level  currently  reflects 
only  the  cost  of  providing  an  adequate  education  in  the  pre-Education  Reform  environment. 
Recently,  the  legislature  appointed  its  own  Foundation  Budget  Review  Conmiission,  which  may 
correct  this  omission. 
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Conclusion 

MERA  enacted  a  tremendously  ambitious  education  policy  agenda  for  the  Commonwealth.  This 
agenda,  which  was  based  on  the  principle  of  standards-based  reform,  fundamentally  altered  the 
relationship  between  state  and  local  education  authorities.  The  state  role  grew,  embracing  not 
only  financial  support  for  local  districts,  but  also  curriculum,  assessment,  school  and  district 
accountability,  data  collection,  recertification  and  professional  development  for  educators,  early 
childhood  education,  school  choice  and  charter  schools,  and  support  for  local  districts 
implementing  new  forms  of  school  govemance. 

In  some  areas,  most  notably  maintenance  of  the  state's  commitment  to  bring  all  districts' 
spending  up  to  the  foimdation  level,  it  has  been  a  success.  In  other  areas,  such  as  data  collection 
and  the  implementation  of  local  government  reforms,  it  has  fallen  short.  The  state's  record  in 
putting  MERA  into  effect  provides  concrete  examples  of  the  political,  organizational,  and 
resource  issues  discussed  in  Part  II. 

Political  issues,  such  as  disagreement  over  how  to  construct  the  collaborative,  shared 
responsibility  system,  the  extent  of  DOE' s  role,  and  the  overall  direction  and  goals  of  Education 
Reform,  complicated  the  state's  task  in  setting  standards,  holding  schools  and  districts 
accountable,  and  enacting  local  govemance  and  management  reforms. 


« 


i 


Organizational  issues,  such  as  communication  and  coordination  among  arms  of  the  state 
government,  within  DOE  itself,  and  between  state  and  local  educational  authorities,  also  limited 
the  state's  implementation  capacity.  The  consequences  have  been  clearest  for  the  areas  of 
accountability,  data  collection  and  use,  and  state-level  policy  making. 

I 
Finally,  it  is  difficult  to  overstate  the  importance  of  resources  as  a  constraint  on  MERA 

implementation.  The  law  was  passed  at  the  point  when  the  Department  of  Education  had  its 

lowest  staff  level  in  years.  Since  1993,  the  size  of  DOE's  staff  has  increased,  but  slowly  and 

modestly.  With  more  financial  resources,  which  would  probably  have  been  spent  mostly  on 

human  resources  (both  DOE  staff  and  contracts  with  people  outside  DOE),  the  Department's 

efforts  in  all  MERA  areas  might  have  been  more  successful. 
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IV.  Recommendations 


Although  assessing  "state  capacity"  seems  at  first  to  be  a  straightforward  matter  of  counting  the 
people  and  resources  currently  devoted  to  a  task  and  asking  whether  or  not  it  is  sufficient,  our 
analysis  has  looked  more  broadly  at  goals,  organizational  elements,  and  stakeholder  perspectives. 
There  is  strong  evidence  that  current  capacity  should  be  strengthened  to  carry  out  even  the 
minimal  "tight-loose"  conception  of  the  state  role  under  Education  Reform.  This  minimal  role 
would  still  involve  the  state  in  collecting  data  and  holding  all  schools  (not  just  the  lowest- 
performing  ones)  accountable  for  their  students'  performance.  These  are  two  areas  in  which  the 
state's  capacity  has  been  both  weak  and  slow  to  develop. 

This  minimal  role,  however,  is  not  enough.  Having  set  high  standards  for  students,  and  begun  to 
implement  the  requirement  that  all  students  must  pass  the  tenth-grade  mathematics  and  English 
language  arts  MCAS  examinations  in  order  to  graduate  firom  high  school,  the  state  must  provide 
more  support  and  guidance  to  local  education  authorities.  For  most  districts,  the  "tight-loose" 
assumption  that  districts  would  be  able  on  their  own  to  determine  how  to  reach  high  standards 
has  been  disconfirmed  by  experience.  DOE  needs  increased  fiinds  and  ability  to  oversee, 
coordinate,  and  support  diverse  activities. 

On  the  basis  of  our  analysis,  we  recommend  that  policy  makers  consider  the  following  changes: 

1 .  Increase  the  funds  available  for  DOE  operations,  and  use  the  Increased  funds 
to  improve  the  agency's  capacity  in  key  areas. 

State  elected  officials  have,  understandably,  held  financial  assistance  to  districts  to  be  their  major 
priority.  For  every  dollar  of  Education  Reform  funds  sent  to  the  Department  of  Education, 
approximately  $85  has  gone  to  school  districts.  Many  of  the  Department  of  Education's 
difficulties  in  implementing  Education  Reform  can  be  traced  at  least  in  part  to  the  relative 
scarcity  of  fimds  to  pay  for  oversight  and  support  activities  by  state  employees  or  by  non-state 
employees  working  on  state  contracts. 

The  state  role  under  MERA  requires  collection  of  data,  use  of  data  to  make  policy  decisions,  and 
development  of  effective  "feedback  loops"  that  communicate  to  policy  makers  whether  policy  is 
working.  DOE  has  particularly  strong  resource  needs  in  the  areas  of  data  collection  and 
management,  research  and  evaluation,  and  assessment.  Other  important  areas  of  need  include 
staff  for  the  school  building  assistance  program  and  funds  to  be  used  as  state  matches  for  federal 
grants. 

2.  Ensure  that  legislatively  mandated  programs  Include  adequate  funds  for  DOE 
to  administer  and  evaluate  them. 

The  legislature  often  mandates  new  programs  without  setting  aside  funds  for  program 
administration  and  evaluation.  This  leaves  DOE  with  the  unenviable  choice  of  either 
administering  state-funded  programs  with  federally  funded  staff,  which  can  leave  the 
Commonwealth  vulnerable  to  federal  audit  exceptions,  or  offering  insufficient  program 
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assistance  and  oversight.  A  concrete  way  of  easing  the  above-mentioned  resources  shortfall 
would  be  for  the  legislature  to  specify  that  a  given  percentage  of  program  fiinds  be  used  for 
legitimate  administrative  costs — ^perhaps  5%  for  administration  and  3%  for  evaluation.  In 
general,  whenever  state  government  creates  a  new  grant  program  or  imposes  a  new  responsibility 
or  reporting  requirement  on  the  DOE  or  local  school  districts,  it  should  specify  a  source  of 
fimding  for  administrative  support,  including  resources  for  facilitation,  oversight,  and  evaluation 
of  the  program. 

3.  Allow  overhead  from  DOE's  Federal  grants  to  go  to  DOE  rather  than  the 
General  Fund. 

Another  area  in  which  DOE  seems  to  be  gaining  administrative  burdens  without  commensurate 
administrative  resources  is  in  the  area  of  Federal  grants.  Primary  administrative  responsibility 
for  these  grants  goes  to  the  Department,  while  the  overhead  funds  go  to  the  Commonwealth's 
General  Fund.  The  state  should  consider  allowing  the  overhead  components  of  federal  grants  to 
remain  with  DOE  rather  than  reverting  to  the  state  general  fund.  Other  sources  of  funds  include 
the  revenues  from  educator  certification  fees,  which  also  currently  go  to  the  General  Fund  rather 
than  Department  administrative  costs. 

4.  Salaries  for  DOE  staff  should  be  increased,  in  order  to  attract  greater  numbers 
of  experienced  educators  to  work  in  the  agency. 

DOE  is  unable  to  compete  with  either  large  school  districts  or  the  private  sector  in.  terms  of 
salaries  offered  for  mid-  to  senior-level  staff.  This  has  a  number  of  harmful  impacts  on  capacity, 
including  reducing  the  pool  of  applicants  for  staff  positions,  which  reduces  expertise  and  greatly 
extends  the  time  required  for  conducting  searches  for  state  staff  This  in  turn  contributes  to  a 
situation  where  hiring  state  staff  is  seen  as  so  difficult  and  time-consuming  that  it  is  actively 
avoided,  and  therefore  to  an  increased  dependence  on  contract  employees.  Contract  employees 
can  make  higher  salaries,  but  they  work  without  benefits,  and  as  a  result  they  turn  over  more 
frequently  than  state  staff,  and  they  generally  take  their  expertise  with  them  when  they  leave.  In 
some  important  cases — a  state  bilingual  education  director  was  mentioned  as  an  example — ^DOE 
has  been  unable  to  fill  positions  at  all,  due  to  limited  pools  of  experts  in  the  field  and  a 
combination  of  greater  responsibiUties  and  lower  pay  at  the  Department. 

This  is  a  difficult  issue  to  resolve,  given  that  the  salary  ranges  for  DOE  are  generally  the  same  as 
for  all  state  agencies.  DOE  has  creatively  used  contracted  employees,  sabbatical  teachers,  and 
other  mechanisms  to  address  this  shortcoming,  but  the  capacity  costs  of  these  practices  cannot  be 
xmderestimated.  Given  the  ambitious  combination  of  support  and  regulatory  roles  required  of 
DOE  by  MERA,  an  inability  to  hire  sufficient  numbers  of  experienced  educators  is  a  significant 
limitation. 
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5.  Increase  involvement  of  and  communication  with  educators  in  the  field  in 
implementation  of  Education  Reform. 

The  DOE  has  frequently  been  criticized  for  taking  an  overly  regulatory  stance  towards  local 
schools  and  districts,  and  for  not  adequately  understanding  the  needs  of  and  demands  on 
educators  in  the  field.  Policy  implementation  in  an  environment  where  power  is  divided  among 
federal,  state,  and  local  authorities  depends  upon  cooperation  at  least  as  much  as  it  does  on  legal 
and  regulatory  compliance.  The  entire  state  govenmient  needs  to  change  the  tone  of  its 
interactions  with  educators  in  the  field  and  work  toward  a  more  collaborative  relationship.  DOE 
has  already  begun  to  take  some  initiative  in  this  area.  Overall,  the  relationship  between  state 
authorities  and  the  field  needs  to  become  less  adversarial.  From  the  governor,  the  legislature,  the 
Board,  the  Department,  and  the  various  education  associations,  there  needs  to  be  some  sort  of 
common  attempt  to  say  that  Education  Reform  is  a  worthy  goal,  that  we  have  much  to  celebrate, 
that  we  are  going  to  get  there,  and  that  we  will  move  forward  together. 

6.  Where  appropriate,  use  resources  and  organizations  outside  of  DOE  to 
expand  state  capacity. 

Not  all  capacity  issues  need  to  be  addressed  by  adding  DOE  staff.  Partnerships  and  alliances 
with  other  public  and  private  entities  should  also  be  considered.  However,  it  is  also  important  to 
recognize  the  limitations  of  contracting  out  as  a  means  of  policy  implementation.  One  DOE 
senior  staff  member,  who  has  worked  extensively  with  contractors,  cautioned  in  an  interview  that 
because  contractors  are  not  necessarily  familiar  with  the  Massachusetts  policy  environment,  their 
work  sometimes  needs  to  be  re-done  or  tailored  by  DOE  staff  As  Elmore,  Abelmann,  and 
Fuhrman  say  of  student  assessment,  "While  contracting  out  allows  states  to  run  performance- 
based  systems  relatively  effectively,  it  also  contributes  to  the  problem. .  .of  a  separation  between 
technical  expertise  about  assessment  and  evaluation  and  the  policy  makers  and  high-level 
administrators  who  are  responsible  for  the  systems"  (1996,  p.  95). 

With  that  caveat,  there  are  some  areas  where  DOE  lacks  capacity  in  which  there  is  likely  to  be 
relevant  expertise  elsewhere.  For  some  areas,  such  as  school  building  assistance  and  school  and 
district  auditing,  that  expertise  may  come  from  other  state  agencies. 

For  other  areas,  assistance  may  come  from  outside  of  state  agencies.  DOE  may  be  able  to 
increase  its  capacity  in  areas  like  evaluation  and  professional  development  by  bringing  together 
others  in  the  state — educators,  professional  development  providers,  consultants,  and  higher 
education — ^to  work  on  specific  projects  where  there  is  need.  DOE  is  currently  moving  in  this 
direction  with  its  development  of  a  network  of  "preferred  providers"  in  professional 
development. 

In  the  area  of  technical  assistance,  DOE  might  build  a  similar  network  to  provide  technical 
assistance  to  schools  identified  as  underperforming.  In  1995,  Texas  moved  the  technical 
assistance  fimction  out  of  its  state  education  agency  into  a  set  of  20  existing  regional  assistance 
centers,  fiinded  by  a  mix  of  federal,  state,  and  local  sources  (Johnston,  1999).    In  Massachusetts, 
DOE  might  establish  a  list  of  preferred  providers  in  these  areas  and  enter  into  contract 
relationships  with  them  to  provide  services  that  DOE  cannot. 
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However,  it  is  not  the  case  that  such  a  network  can  be  built  with  no  increase  in  DOE  financial, 
human,  or  technological  resources.  DOE  needs  such  resources  to  stimulate,  organize,  and 
manage  participation  in  the  network  and  to  monitor  the  quality  of  participants'  work.  DOE 
employees  would  be  needed  to  ensure  that  service  providers  carry  out  appropriate  services  and  to 
safeguard  the  state's  interests,  and  continued  investment  in  data  collection  and  information 
technology  networks  would  be  needed  in  order  to  enable  communication  within  the  network  and 
evaluation  of  programs  and  poHcies. 

7.  Resolve  uncertainty  over  responsibility  for  accountability,  monitoring,  and 
oversiglit. 

Over  the  past  several  months,  uncertainty  over  the  division  of  authority  between  the  DOE  and 
the  new  accountabiUty  board  in  the  Governor's  Office  has  been  an  obstacle  to  capacity-building 
at  DOE.  Statewide,  because  understanding  of  goals  is  a  key  element  of  capacity,  the  lack  of 
clarity  over  accountability  goals  and  functions  has  undermined  capacity.  In  addition  to 
establishing  a  clear  sense  of  which  roles  will  be  played  by  the  Governor's  Office  and  which  by 
the  DOE,  it  is  also  necessary  to  determine  how  both  entities'  work  fits  with  that  of  the  State 
Auditor  and  the  Education  Reform  Review  Commission.  Finally,  lessons  may  be  learned  in  the 
area  of  district  accountabiUty  by  comparing  the  different  accountability  methods  used  to  evaluate 
charter  schools  and  regular  public  schools. 

8.  Maintain  and  enhance  DOE's  capacity  to  play  a  regulatory  role. 

Although  regulatory  compliance  is  not  a  popular  function  of  DOE,  it  is  a  key  part  of  its  work  as  a 
state  agency.  Emphasis  on  DOE  as  a  supporter  or  facilitator  of  improved  teaching  and  learning 
must  be  seen  as  complementing  rather  than  supplanting  its  role  as  the  enforcer  of  federal  and 
state  regulations.  Although  networks  outside  the  agency  can  assist  in  the  support  and  facilitating 
functions  and  in  service  delivery,  legal  comphance  remains  the  DOE's  legal  responsibility  as  a 
state  agency. 

9.  Expand  the  use  of  sampling  in  data  collection  and  program  evaluation. 

Closely  monitoring  the  activities  of  371  districts  and  1,874  schools  is  a  dauntmg  task,  but  in 
many  cases  a  sampling  strategy  can  deliver  representative  data  at  lower  capacity  costs  than 
collecting  data  from  all  districts  and/or  schools.  DOE  already  uses  this  type  of  sampling 
approach  in  its  Coordinated  Review  Program,  and  there  may  be  numerous  other  data  collection 
areas  in  which  a  sampling,  rather  than  a  census,  approach  should  be  considered.  Adequate  data 
collection  and  analysis  through  a  sampling  strategy  would  probably  still  require  additional  DOE 
capacity,  but  less  than  if  the  whole  state  were  to  be  audited  each  year. 
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10.  Improve  coordination  and  communication  within  DOE. 

Many  interviewees  (inside  and  outside  DOE)  talked  about  their  impression  that  the  work  of 
various  DOE  clusters  and  staff  members  is  not  well  coordinated,  both  because  there  isn't  a  chief 
operating  officer  or  staff  member  with  similar  knowledge  of  and  authority  over  which  cluster  is 
doing  which  tasks  and  because  communication  among  clusters  is  largely  ad  hoc.  Since  we  began 
our  interviews,  the  Commissioner  has  embarked  upon  a  reorganization  that  addresses  one  of 
these  issues  by  adding  a  chief  operating  officer  to  the  organizational  chart.  This  reorganization, 
and  subsequent  ones,  may  also  make  cross-cluster  communication  and  collaborations  more 
routine. 

11. Consider  regional  or  at  least  western  and  southeastern  Massachusetts  field 
offices. 

At  present,  all  of  DOE,  including  the  Program  Quality  Assurance  staff,  whose  areas  of 
responsibility  are  geographically  defined,  are  based  in  Maiden.  The  old  regional  structure  of 
DOE  was  eliminated  between  1988  and  1991,  as  an  effort  to  cut  costs,  to  respond  to  charges  that 
the  regional  offices  were  overly  regulatory  in  their  interactions  with  schools  and  districts,  and  to 
unify  interpretations  of  laws  and  regulations  under  the  Commissioner's  authority. 

The  risk  in  reinstating  some  level  of  regional  structure  for  DOE  is  that  the  problems  of  the  old 
system  would  also  be  reinstated.  The  benefit  would  be  a  greater  sense  of  connection  to  the 
agency  for  school  and  district  staff  in  the  parts  of  the  state  that  are  the  furthest  firom  the  Boston 
area.  As  our  survey  suggests,  educators  who  have  more  regular  contact  with  DOE  have  more 
positive  impressions  of  Education  Reform.  Creating  regional  offices  would  also  make  it  easier 
for  DOE  staff  to  spend  more  time  in  schools  and  districts. 

State  policy  makers  should  go  beyond  framing  the  issue  as  a  choice  between  maintaining 
centralization  and  instituting  decentralization.  Regionally  based  staff  need  not  become 
disconnected  from  Maiden,  but  could  instead  use  information  technology  for  intemal  and 
external  communication  and  coordination. 

12. Conduct  additional  research. 

The  research  capacity  of  the  state,  inside  and  outside  DOE,  needs  to  be  built  up,  either  through 
intemal  research  staff  or  through  additional  resources  for  contracted  research.  One  potential 
model  is  the  Consortium  on  Chicago  School  Research,  described  in  Appendix  1 .  Areas  of 
needed  research  include  the  following: 

•  How  are  districts  spending  their  MERA  funds? 

•  What  would  a  data  management  system  optimally  look  like,  based  on  the  data-use  needs  of 
both  the  Department  and  local  educators? 

•  What  are  the  best  practices  in  professional  development  now,  and  how  could  a  provider 
network  be  organized  to  bring  these  practices  into  classrooms  across  the  state? 

•  Evaluation  of  early  childhood  programs  and  spending — what's  the  best  use  of  state  fimds? 

•  Is  there  a  fiscal  crisis  coming,  due  to  the  large  number  of  current  school  building  projects, 
that  will  drain  instructional  resources  in  the  future? 
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*  *  * 


When  MERA  was  passed  in  1993,  its  proponents  had  a  vision  of  a  future  in  which  all 
Massachusetts  children  would  have  equal  educational  opportunities  and  would  be  able  to  meet 
world-class  academic  standards.  The  path  to  that  future  has  proved  long  and  challenging. 
Several  of  the  state  policy  makers  we  met  with  described  Education  Reform  implementation  as 
having  recently  reached  "the  hard  part,"  or  as  passing  through  "adolescence."  They  remained 
hopefiil,  however,  that  the  challenges  of  Education  Reform  can  be  surmounted.  We  hope  that  the 
analysis  and  recommendations  we  have  presented  in  this  paper  will  be  used  constructively  to 
support  the  Commonwealth  in  meeting  these  challenges. 
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Appendix  1 

Information  About  Programs  in  Other  States 

1.  The  Kentucky  Department  of  Education  and  the  Kentucky  Education  Reform 
Act 

In  1989,  the  Kentucky  Supreme  Court  declared  inRose  v.  Council  for  Better  Education  that  the  state's  entire  system 
of  public  education  was  unconstitutional.  The  Court  ordered  the  state  legislature  to  create  a  new  system  that  would 
deliver  an  adequate  education  for  every  child  in  the  state,  with  "adequate"  defined  in  terms  of  educational  standards. 

Spurred  by  the  court  order,  the  Kentucky  General  Assembly  passed  the  Kentucky  Education  Reform  Act  (KERA) 
the  following  year.  KERA  included  funding  increases  for  public  education  coupled  with  systemic  school  reform. 
The  guiding  philosophy  behind  the  reform  was  that  schools  should  be  freed  from  excessive  regulation  and  oversight 
but  held  accountable  for  results  (Lusi,  1997,  p.  29).  The  KERA  was  one  of  the  models  for  the  Massachusetts 
Education  Reform  Act. 

Despite  the  similarity  in  philosophy  between  the  two  reform  packages,  the  KERA  had  different  consequences  for  the 
capacity  of  the  Kentucky  Department  of  Education  (KDE)  than  the  MERA  has  had  for  the  Massachusetts 
Department  of  Education.  The  KDE  was  completely  reorganized  in  the  year  following  KERA,  with  no  guarantees 
that  staff  would  be  able  to  remain  employed  with  the  agency.  All  of  the  KDE's  staff  positions  were  terminated  as  of 
June  30,  1991,  and  the  reorganized  department  opened  on  July  1,  1991.  Fewer  than  20%  of  deputy  and  associate 
commissioners  and  division  directors  from  the  old  KDE  continued  their  employment  in  the  new  one;  however,  just 
over  80%  of  the  civil  service  employees  stayed  on  in  new  positions  (Lusi,  1997,  p.  31).  The  new  KDE  had  40  fewer 
staff  positions  than  the  old  one  (Van  Meter,  1992). 

Even  with  this  cut  in  positions,  KDE  remains  larger  than  the  Massachusetts  DOE  both  in  terms  of  absolute  number 
of  employees  and  relative  to  the  total  size  of  the  state's  public  school  system.  Massachusetts  has  976,000  students, 
371  districts,  1,874  schools,  71,900  teachers,  and  405  FTE  staff  in  the  Department  of  Education.  In  contrast, 
Kentucky  has  637,000  students,  176  districts,  1,346  schools,  39,800  teachers,  and  865  FTE  Department  of  Education 
staff.  With  double  the  number  of  FTE's  in  the  state  education  agency  serving  smaller  numbers  of  students,  districts, 
schools,  and  teachers,  the  KDE  seems  to  have  greater  internal  resources  than  the  Massachusetts  Department  of 
Education. 

Apart  from  staffing  levels,  the  major  organizational  difference  between  the  two  agencies  is  that  unlike  the 
Massachusetts  Department  of  Education,  the  reorganized  KDE  includes  regional  centers  with  responsibilities  in 
professional  development,  school  and  district  consolidated  planning,  technical  assistance,  program  design  and 
development,  and  local  capacity  building.  The  Kentucky  regional  centers  assist  schools,  districts,  and  consortia  in 
planning  professional  development;  identify  and  remediate  gaps  in  professional  development,  in  collaboration  with 
partners;  provide  technical  assistance  in  development,  evaluation,  and  updating  of  school  and  district  transformation 
plans;  assist  districts  and  schools  in  standards-based  curriculum  development;  identify  appropriate  and  useful 
resources  for  technical  assistance;  and  supply  technical  information  for  implementing  instructional  strategies;  assist 
in  design  and  development  of  KDE  curriculum  materials;  ensure  involvement  of  local  educators  in  KDE  initiatives; 
and  provide  leadership  training  (KDE,  2001). 

Another  difference  that  affects  staff  capacity  is  that  the  KDE  may  go  outside  normal  state  hiring  guidelines  for  up  to 
50%  of  its  employees  below  the  Division  Director  level.  KDE  does  this  through  Memoranda  of  Agreement  (MOA) 
with  local  school  districts  and  the  Ohio  Valley  Education  Cooperative.  When  KDE  wants  to  "borrow"  a  district 
employee  for  two  or  three  years,  as  for  the  Highly  Skilled  Educators  program  (see  description  of  the  Highly  Skilled 
Educators  program  below),  it  enters  into  an  MOA  with  the  district  and  pays  the  employee's  regular  district  salary. 
When  KDE  wishes  to  hire  a  staff  member  who  is  not  a  district  employee,  it  arranges  to  have  the  Ohio  Valley 
Education  Cooperative  hire  the  person,  and  then  enters  into  the  same  sort  of  MOA.  This  system  has  the  positive 
effect  of  increasing  the  state's  flexibility  in  hiring  experienced  educators  and  other  desirable  personnel;  however,  it 
also  is  no  faster  than  the  usual  state  process  and  creates  inequities  within  KDE  and  between  KDE  and  other 
agencies. 
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2.  Comparing  SEA'S 

Although  comparing  the  size  of  state  education  agencies  (SEAs)  sounds  as  if  it  should  be  a  simple  matter  of  getting 
figures  on  each  state's  SEA  budget,  number  of  FTE's,  and  total  enrollment,  in  truth  the  comparison  is  much  more 
complicated.  In  different  states,  SEA's  have  different  combinations  of  responsibilities,  including  administration  of 
various  federal  programs,  complicating  comparative  analysis.  After  contacting  organizations  such  as  the  Council  of 
Chief  State  School  Officers,  the  Education  Commission  of  the  States,  a.nd  Education  Week,  we  also  learned  that 
there  is  currently  no  source  of  comprehensive  basic  data  on  budgets  and  FTE's.    Despite  the  lack  of  comparable 
data,  a  few  generalizations  can  be  made: 

•  SEA  budgets  and  staff  began  to  decrease  in  the  1980s  and  1990s,  ironically  as  many  state  boards  and 
legislatures  were  taking  the  lead  in  standards-based  reform.  According  to  Education  Week,  at  least  27  SEAs  had 
fewer  employees  in  1998  than  they  had  had  in  1980  (Johnston,  1999).  Massachusetts  fits  this  trend,  with  most  of  the 
loss  of  SEA  staff  occurring  during  the  1980s.  In  1980,  the  DOE  had  990  employees.  In  1991,  it  had  452.  More 
state-fiinded  (374)  than  federally-fiinded  (164)  positions  were  lost.  As  a  result,  the  DOE  went  fi-om  having  63%  of 
its  FTEs  state-funded  to  having  55%  state-funded.  This  shift  matters,  because  the  state  has  more  flexibility  in  how  it 
deploys  state-funded  employees  to  implement  state  policy  initiatives. 

•  Although  federal  sources  make  up  only  about  7%  of  the  nation's  total  education  spending,  federal  dollars 
are  an  important  source  of  support  for  SEAs.  According  to  a  1994  report  by  the  U.S.  General  Accounting  Office, 
the  average  SEA  gets  41%  of  its  funding  and  43%  of  its  FTEs  firom  federal  programs.  In  any  given  state,  the 
percentage  of  staff  and  funding  firom  federal  sources  varies  according  to  which  federal  programs  the  SEA 
administers,  firom  about  10%  to  about  80%  (GAO,  1994).  In  Massachusetts,  45%  of  DOE  FTEs  are  federally 
funded. 

•  More  state  spending  does  not  necessarily  mean  more  state  bureaucracy.  SEAs  pass  through  nearly  all  of 
the  federal  and  state  funds  they  receive  to  local  schools  and  districts,  with  very  little  held  at  the  state  level  for 
administration  of  programs.  In  1994,  SEAs  passed  through  95%  of  the  federal  money  and  99%  of  the  state  money 
they  received  (GAO).  This  pattern  holds  true  for  the  Massachusetts  DOE  and  Education  Reform  funds. 

3.  The  Rhode  Island  Department  of  Education's  School  Accountability  for 
Learning  and  Teaching  Program 

In  the  1997-1998  school  year,  Rhode  Island  began  a  new  comprehensive  accountability  system  for  schools.  Like 
Massachusetts,  Rhode  Island  sets  improvement  targets  for  schools  and  has  the  power  to  intervene  in  failing  schools. 
Despite  this  broad  similarity,  however,  Rhode  Island's  School  Accountability  for  Learning  and  Teaching  Program 
(SALT)  is  quite  different  from  the  approach  to  accountability  taken  for  most  public  schools  in  Massachusetts. 

SALT  is  intended  to  provide  support  to  local  schools  as  they  conduct  self-studies  of  their  improvement  plans.  Each 
school  undertakes  the  SALT  process  every  five  years.  The  process  consists  of  school-level  development  of  an 
Improvement  Plan,  with  involvement  by  parents,  teachers,  and  community  members,  school  reporting  to  the  local 
community,  parents,  and  teachers,  a  school  visit  by  state  authorities,  and  state  support  and  intervention.  The  SALT 
process  is  more  like  the  Massachusetts  charter  school  evaluation  process  than  like  the  regular  accountability  process 
in  Massachusetts,  because  it  involves  every  school  rather  than  only  the  ones  identified  as  under-performing  on  the 
basis  of  their  test  scores.  To  support  the  local  parts  of  the  SALT  process,  guides  have  been  developed  by  Department 
of  Education  staff  to  assist  in  the  process. 

At  the  state  level,  SALT  is  staffed  by  fourteen  people:  a  Director,  a  Program  Coordinator,  a  Coordinator  for 
Training,  an  Assistant  Trainer,  a  Logistics  Administrator,  three  consultants  that  edit  visit  reports  and  maintain  the 
SALT  website,  and  six  educators  on  two-year  Fellowships  from  school  districts.  Each  Fellowship  last  for  two  years, 
and  each  Fellow  is  responsible  for  six  visits  annually.  The  state  site  visit  teams  consist  of  members  of  the 
community  and  local  school  staff  and  faculty.  There  is  an  application  and  interview  process  for  all  team  members. 
Each  site  visit  lasts  four  days  and  the  team  size  is  dependent  on  the  size  of  the  school. 
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Statewide,  Rhode  Island  has  about  300  schools,  compared  with  about  1,900  in  Massachusetts  (the  Rhode  Island 
figure  is  from  Education  Week's  Quality  Counts;  the  Massachusetts  figure  is  a  revised  number  provided  by  a  DOE 
employee),  which  means  that  a  school  accountability  apparatus  with  the  same  functions  in  Massachusetts  as  in 
Rhode  Island  would  need  to  be  about  six  times  larger  than  Rhode  Island's — about  84  people. 

4.  The  Kentucky  Department  of  Education's  Highly  SIciiled  Educators  Program 

In  Kentucky,  schools  that  are  identified  as  in  need  of  assistance  based  on  their  assessment  scores  receive  help  from 
teachers  and  administrators  under  the  Department  of  Education's  Highly  Skilled  Educators  Program. 

In  the  early  years  of  Kentucky  Education  Reform  Act  implementation,  the  Distinguished  Educators  Program  (as  it 
was  then  known)  was  badly  understaffed  because  of  a  budget  cut  made  by  the  state  legislature.  As  a  result  of  the 
cut,  23  Distinguished  Educators  served  150  schools,  which  had  the  discretion  to  choose  not  to  participate  in  the 
program  (Elmore,  Abelmann,  and  Fuhrman,  p.  83).  Each  Distinguished  Educator  worked  with  multiple  schools  and 
thus  could  not  pay  full  attention  to  any  one  of  them  (Calvert,  Gaus,  and  Ruscoe,  2000). 

This  year,  the  program's  total  budget  for  operations  and  personnel  is  $23  million,  and  it  serves  61  schools. 
The  Highly  Skilled  Educators  program  is  a  branch  of  the  Division  of  School  Improvement,  which  in  turn  is  part  of 
the  Office  of  Leadership  and  School  Improvement.  The  program  is  staffed  by  a  Branch  Manager,  a  consultant,  and 
an  administrative  assistant.  There  are  63  Highly  Skilled  Educators,  district-based  personnel  who  work  for  the 
Department  of  Education  temporarily  under  Memoranda  of  Agreement  with  their  home  districts  (see  above 
description  of  the  Kentucky  Department  of  Education).  State  funds  pay  the  Highly  Skilled  Educators'  salaries, 
which  are  not  subject  to  the  regular  state  scale  because  of  the  Memoranda  of  Agreement. 

5.  The  Consortium  on  Chicago  School  Research,  Chicago,  IL 

In  1988,  the  Illinois  state  legislature  passed  a  law  that  decentralized  much  of  the  authority  over  Chicago's  public 
schools  to  Local  School  Councils  at  each  school.  The  Chicago  Public  Schools  did  not  have  the  capacity  to  evaluate 
the  implementation  and  effects  of  decentralization  itself,  and  so  asked  Professor  Anthony  S.  Bryk  of  the  University 
of  Chicago  to  convene  a  research  collaboration  between  several  grass-roots  education  reform  advocacy 
organizations  and  researchers  at  Chicago  area  universities.  The  Consortium  on  Chicago  School  Research  provides  a 
model  both  of  a  research  and  data-gathering  organization  and  of  collaboration  between  public  education  authorities, 
commxmity  groups,  foimdations,  and  higher  education. 

The  Consortium  produces  reports  and  conducts  briefings,  with  the  goal  of  advancing  discussion  of  school 
improvement  and  school  reform.  Its  work  has  five  general  audiences:  key  decision  makers  in  the  school  system, 
city,  and  state;  business  leaders,  education  advocacy  leaders  and  organizations,  and  community  organizations; 
members  of  the  local  school  communities;  the  people  of  greater  Chicago;  and  the  national  policy  and  research 
community.  Research  activities  include  a  biannual  survey  of  students,  teachers,  and  principals.  Students  are  asked 
about  their  experiences  in  school,  their  attitudes  about  school,  and  the  activities  they  undertake  in  their  schools. 
Principals  and  teachers  are  asked  about  instruction,  professional  development,  and  their  schools'  characteristics  as 
workplaces.  The  Consortium  also  maintains  an  archive  of  data  provided  by  the  Chicago  Public  Schools,  including 
test  scores  and  other  information  on  individual  students,  which  can  be  matched  with  their  survey  responses.  The 
archive  also  includes  demographic,  certification,  and  educational  preparation  information  on  the  city's  teachers,  as 
well  as  data  on  the  communities  in  which  the  schools  are  located. 

Examples  of  recent  Consortium  reports  include  Instruction  and  Achievement  in  Chicago  Elementary  Schools, 
Authentic  Intellectual  Work  and  Standardized  Tests:  Conflict  or  Co-Existence? ,  School  Instructional  Program 
Coherence:  Benefits  and  Challenges,  and  an  updated  report  on  the  effects  of  ending  social  promotion.  These  may 
be  downloaded  from  the  Consortium's  website:  http://www.consortium-chicago.org. 

The  Consortium's  Steering  Committee  includes  representatives  from  the  Chicago  Board  of  Education,  the  Chicago 
Public  Schools'  Chief  Executive  Officer  and  Accountability  Office,  as  well  as  from  the  Chicago  Academic  Advisory 
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Council,  the  Illinois  State  Superintendent  of  Schools  office,  the  North  Central  Regional  Educational  Laboratory,  the 
Chicago  Principals  and  Administrators'  Association,  the  Chicago  Teachers'  Union,  Leadership  for  Quality 
Education,  Cross-City  Campaign  for  Urban  School  Reform,  Designs  for  Change,  Northeastern  Illinois  University, 
Northwestern  University,  Chicago  State  University,  Roosevelt  University,  DePaul  University,  the  University  of 
Illinois  at  Chicago,  and  National-Louis  University.  The  Constituent  Advisory  Board  involves  educational  agency 
leaders,  legislators,  aldermen,  foundation  representatives,  principals,  teachers,  other  school  staff,  and  local  school 
council  members. 

In  addition  to  its  seven  directors,  five  of  whom  are  also  university  faculty,  the  Consortium  has  a  large  staff  The 
website  lists  28  full  and  part-time  employees,  including  three  postdoctoral  fellows  (two  from  the  Spencer 
Foundation's  postdoctoral  program),  four  research  assistants  (two  part-time),  a  microsystems  support  specialist,  a 
psychometrician,  a  project  coordinator,  a  survey  coordinator,  a  director  of  fieldwork,  and  a  project  fieldwork  and 
data  coordinator. 

The  Consortium's  core  budget  of  approximately  $1.25  million  per  year  comes  entirely  from  foundation  grants, 
although  early  on  it  received  a  small  amount  of  money  from  the  Illinois  State  Department  of  Education.  Its  three 
main  funders  are  the  Joyce  Foundation,  the  MacArthur  Foundation,  and  the  Spencer  Foundation. 

6.  The  Connecticut  State  Department  of  Education's  Beginning  Educator  Support 
and  Training  Program 

In  1986,  the  Connecticut  General  Assembly  passed  the  Education  Enhancement  Act.  The  goal  of  the  Act  was  to 
enhance  the  professional  status  and  skills  of  the  state's  teaching  force,  by  raising  salaries,  requiring  teachers  to 
participate  in  continuing  education,  replacing  lifetime  certification  with  recertification  every  five  years,  and  by 
requiring  new  teachers  to  demonstrate  mastery  of  academic  skills,  content,  and  pedagogy  through  the  Beginning 
Educator  Support  and  Training  Program  (BEST). 

BEST  consists  of  an  assessment  component  and  a  support  component.  The  support  activities  include  school-based 
mentors  for  new  teachers,  clinics  for  beginning  teachers,  and  content- focused  support  seminars  led  by  experienced 
teachers.  The  assessment  component  consists  of  a  content-specific  teaching  portfolio  for  second-year  teachers. 

The  Connecticut  State  Department  of  Education  (CSDE)'s  capacity  to  develop  and  operate  the  BEST  program  was 
the  result  of  efforts  to  build  both  the  agency's  internal  capacity  in  the  areas  of  research,  evaluation,  and  assessment 
and  its  external  capacity  through  partnerships  with  educators  in  the  field  and  with  agencies  in  other  states  through 
the  Interstate  New  Teacher  Assessment  and  Support  Consortium  (INTASC)  (Fisk,  1998). 

The  BEST  Director,  Catherine  W.  Fisk,  estimates  that  typical  development  costs  for  a  system  like  BEST  would  be 
slightly  more  than  $1  million,  spread  over  three  years,  per  content  area  portfolio.  The  major  component  of  this  cost 
is  salary  and  benefits  for  3.5  FTE  employees  in  the  state  education  agency.  According  to  Fisk,  a  state  education 
agency  could  conceivably  save  money  by  adopting  or  adapting  an  existing  portfolio  assessment  from  another  state, 
although  there  would  still  be  costs  associated  with  validation  and  standard  setting.   Following  development  of  the 
system,  implementation  costs  would  be  about  S3. 6  million  per  year  (Fisk,  1997,  pp.  12-15). 

Currently,  BEST  is  managed  by  9  FTE  staff  at  the  CSDE  (3.5  in  assessment  development,  validation,  and  program 
evaluation,  plus  5.5  in  operations)  plus  8  FTE  teachers-in-residence  at  CSDE,  whose  role  is  to  develop  and 
implement  content-specific  portfolio  assessment  and  training  for  mentors,  scorers,  and  beginning  teachers. 

BEST  also  involves  3.1  FTE  staff  at  the  state's  six  Regional  Educational  Service  Centers  (RESCs).  The  RESCs, 
which  are  similar  to  the  regional  collaborative s  in  Massachusetts,  were  established  outside  of  state  government  by 
cooperating  local  boards  of  education  during  the  1960s  and  ultimately  given  official  recognition  by  the  state 
government.  RESCs  are  governed  by  boards  representing  each  of  their  participating  boards  of  education,  and  their 
fiinding  comes  from  a  mixture  of  public  and  private  sources.  The  RESC  staff  involved  in  BEST  provide 
district/facilitator  support,  general  district  support,  regional  support,  training,  and  program  monitoring  (RESC  Field 
Staff  Responsibilities,  2000).  One  of  the  RESCs,  EASTCONN,  currently  serves  as  the  central  contractor  for  BEST. 
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Its  role  is  to  implement  and  coordinate  statewide  training,  portfolios,  and  budget  matters.  In  2000-01,  EASTCONN 
budgeted  for  471  training  sessions,  including  285  for  new  teachers. 

For  FY  2000-01,  BEST'S  operating  budget  is  approximately  $3.5  million.  This  figure  does  not  include  state  agency 
staff  and  teachers-in-residence,  but  does  include  contractor  staffing,  payments  to  districts,  and  assessment  scoring. 
The  budget  assumes  that  3500  first  year  teachers  and  2800  second  year  teachers  will  be  served  by  3500  mentors  and 
930  assessors.  There  will  be  215  beginning  teacher  clinics  and  seminars,  26  initial  mentor  training  sessions,  44 
update  mentor  training  sessions,  and  16  assessor  training  sessions.  Beginning  teachers  participate  for  2  or  3  years. 
Because  the  state  is  currently  in  the  last  stages  of  portfolio  development  and  validation,  staffing  and  budget  will 
decrease  slightly  starting  in  FY  2002-03. 
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Appendix  2 

Analysis  of  Survey  Data: 

State  Capacity  to  Perform  Education  Reform  Responsibilities 


Introduction 

The  goal  of  the  survey  portion  of  this  study  is  to  examine  the  capacity  of  the  Commonwealth  from  the  point  of  view 
of  the  "end  users" — in  this  case,  the  local  educational  personnel  who  are  the  ground-level  implementers  of  education 
reform.  Therefore,  in  addition  to  document  analysis  and  state-level  stakeholder  interviews,  local  educational 
leaders'  assessments  of  whether  or  not  the  Commonwealth  has  sufficient  capacity  to  implement  MERA  were  also 
examined  through  the  use  of  survey  research.  For  the  purposes  of  this  study,  local  educational  leaders  were  defined 
as  superintendents  (or  a  representative  from  a  central  district  office),  principals/assistant  principals,  and  teachers. 
Nearly  identical  surveys^  were  sent  to  a  sample  representing  each  of  these  key  local  stakeholder  groups  throughout 
the  Commonwealth. 

The  following  report  briefly  summarizes  the  findings  from  this  survey  research.  Most  of  the  analyses  are  descriptive 
in  nature,  with  a  few  analysis  (mainly  in  the  form  of  factor  analysis  and  simple  correlations)  of  relationships  among 
variables  being  conducted,  along  with  some  mean  comparisons  to  see  if  there  were  differences  between  the  three 
groups  -  superintendents,  principals,  and  teachers  -  targeted  for  this  survey  research.  If  you  are  interested  in  details 
of  the  analysis,  please  contact  Joseph  Berger  of  the  Center  for  Education  Policy  atjbberger@educ.umass.edu. 

Description  of  Sample 

Survey  instruments  used  in  this  study  added  to  the  overall  data  collection  on  goals,  organizational  elements,  and 
perspectives.  In  the  area  of  goals,  respondents  were  asked  to  share  their  impressions  of  the  state's  goals  under 
Education  Reform  and  how  well  those  goals  fit  with  local  and  school-level  initiatives.  The  survey  also  includes 
opportunities  for  respondents  to  rank  the  importance  of  the  ten  main  Education  Reform  goals  we  have  identified.  In 
the  area  of  organizational  elements,  local  educators  were  asked  to  comment  on  the  data  they  share  with  the 
Department  of  Education  and  the  uses  the  Department  makes  of  that  data.  They  were  also  asked  about  which  parts 
of  the  Department  they  interact  with  most  regularly  and  ask  them  to  evaluate  their  interactions.  Finally,  in  the  area 
of  perspectives,  local  educators'  sense  of  the  appropriate  balance  between  state  and  local  roles  in  education  policy 
was  investigated. 

Surveys  were  sent  out  to  three  groups  of  local  educators  -  superintendents,  principals,  and  teachers.  The 
Massachusetts  Association  of  School  Superintendents,  the  Massachusetts  Elementary  School  Principals  Association, 
the  Massachusetts  Association  of  Secondary  School  Principals,  the  Massachusetts  Teachers  Association,  and  the 
Massachusetts  Federation  of  Teachers  all  supported  the  survey  component  of  this  project  by  urging  their  members  to 
participate  and  by  sharing  mailing  lists. 

We  sent  surveys  to  all  members  of  the  Massachusetts  Association  of  School  Superintendents.  For  principals  and 
teachers,  we  first  developed  a  representative  sample  of  150  school  districts,  based  on  district  size,  DOE's 
classification  of  the  community  type,  geographic  region,  vocational-technical  schools,  involvement  with  the 
McDuffy  suit,  amount  of  Chapter  70  aid  received,  and  whether  or  not  the  district's  School  Committee  publicly 
supported  the  Massachusetts  Association  of  School  Committees'  resolution  to  eliminate  MCAS  as  a  graduation 
requirement.  We  drew  our  samples  of  principals  and  teachers  from  this  sample  of  150  districts. 


^  Wording  was  changed  on  some  items  to  reflect  the  different  levels  of  responsibility  (district,  school  or  classroom) 
of  the  targeted  respondents.  See  Appendix  4  for  copies  of  surveys. 
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For  principals,  we  sent  surveys  to  a  mix  of  high  school,  middle  school,  and  elementary  school  principals.  We 
received  a  mailing  list  from  the  Massachusetts  Secondary  School  Administrators'  Association  (MSSAA)  and  sent 
surveys  to  117  high  school  and  86  middle  school  principals  in  our  selected  districts.  For  elementary  school 
principals,  we  generated  a  mailing  list  for  our  selected  districts  from  a  list  purchased  from  Market  Data  Retrieval,  a 
division  of  Dun  &  Bradstreet,  and  sent  out  surveys  to  331  principals. 

For  teachers,  we  received  mailing  lists  from  the  Massachusetts  Teachers'  Association  and  the  Massachusetts 
Federation  of  Teachers  for  our  150  selected  districts.  For  most  districts,  we  sent  out  surveys  to  6  randomly  selected 
teachers,  distributed  2  each  in  high  school,  middle  school,  and  elementary  school.  In  the  larger,  urban  districts  we 
over-sampled,  mailing  20  surveys  to  Springfield,  Worcester,  and  Lawrence  teachers,  and  60  surveys  to  Boston 
teachers. 

Superintendents  had  a  response  rate  of  41%  (1 13  of  277  surveys  returned).  534  surveys  were  sent  to  principals  and 
167  of  them  were  returned  in  time  for  data  analysis,  resulting  in  a  return  rate  of  31%.  Teachers  had  the  lowest 
response  rate;  98  out  of  6 1 6,  or  1 6%,  were  returned. 


Data  Analysis 

Several  statistical  techniques  were  used  to  analyze  the  survey  data.  The  experience  items  were  factor  analyzed  to 
examine  and  identify  underlying  patterns  of  commonality  among  the  items.  The  exploratory  factor  analysis  was 
conducted  using  an  orthogonal  varimax  rotation  given  that  there  was  no  a  priori  assumption  of  strong  inter- 
correlations  among  the  underlying  factors.  The  factors  were  identified  using  a  0.30  cut-off  score  for  the  factor 
loadings  and  the  factors  were  confirmed  through  the  use  of  Cronbach's  alpha  reliability  analysis. 

Descriptive  statistics  -  frequencies  and  means  -  were  used  to  provide  an  overview  of  how  the  superintendents, 
principals,  and  teachers  collectively  responded  to  the  items  on  the  survey.  In  addition,  independent  samples  t-tests 
were  used  to  identify  differences  in  response  patterns  among  the  three  groups  of  local  educators.  Finally, 
correlational  analysis  was  also  used  to  examine  relationships  among  the  variables. 

Descriptive  Analysis 

The  following  sub-sections  of  the  report  provide  summaries  of  how  district  and  school-level  personnel  perceive 
issues  related  to  the  state's  capacity  to  implement  various  components  of  educational  reform.  The  main  text  of  this 
report  provides  a  narrative  summary  of  the  results  of  the  statistical  analyses  used  to  examine  the  data. 


Overview  of  Experiences 

In  this  section  of  the  survey,  a  sample  of  superintendents,  principals,  and  teachers  were  asked  to  respond  to  a  series 
of  questions  about  the  ways  in  which  they  have  experienced  educational  reform.   More  specifically,  respondents 
were  asked  to  indicate  the  extent  to  which  they  agreed"*  with  twenty  statements  regarding  various  aspects  of  their 
own  experiences  with  the  implementation  of  educational  reform  in  the  Commonwealth. 


Measured  by  a  four  point  indicator  with  responses  that  ranged  from  strongly  disagree  to  strongly  agree 
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The  first  step  in  the  analysis  was  to  examine  the  data  to  see  if  there  were  common  underlying  patterns  of  response  to 
various  items  .  Five  groups  of  items  were  identified  as  having  common  patterns  of  response  among  survey 
respondents.  The  common  groups  of  items  were  labeled  as  follows: 

1)  Local  Impact 

2)  Communication  of  Goals 

3)  Local  Control 

4)  Use  of  Data 

5)  State  Personnel 

The  following  paragraphs  provide  a  descriptive  overview  of  how  survey  respondents  perceive  their  own  experiences 
in  each  of  these  five  areas. 


Local  Impact 

Six  items  focused  on  the  ways  in  which  respondents  perceived  that  educational  reform  has  had  an  impact  on  local 
districts  and  schools  in  the  areas  of  instructional  quality,  curricular  quality,  professional  development,  local 
priorities,  and  overall  district  functioning.  Table  1  displays  the  patterns  of  responses  for  each  of  these  six  items. 

Table  1  -  Response  Patterns  for  Items  Measuring  Experience  with  Local  Impact  of 
Education  Reform. 


Type  of  Experience 

Strongly" 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

Education  reform  has  had  a  positive  impact  on  my 
district 

21.2% 

43.9% 

28.8% 

3.4% 

Education  reform  has  had  a  positive  impact  on  my 
school 

24.9% 

42.7% 

27.4% 

2.8% 

Education  reform  has  improved  the  quality  of 
instruction  in  my  school/district 

13.1% 

49.7% 

25.6% 

6.9% 

Education  reform  has  improved  the  quality  of 
curriculum  in  my  school/district 

17.2% 

52.4% 

19.7% 

7.2% 

Education  reform  has  improved  the  quality  of 
professional  development  in  my  school/district 

21.7% 

51.3% 

17.5% 

6.4% 

Education  reform  fits  well  with  my 
district/school's  other  priorities 

9.8% 

57.9% 

21.1% 

5.6% 

The  findings  reported  in  Table  1  suggest  that  survey  respondents  do  not  generally  believe  that  educational  reform 
has  had  a  positive  impact  on  local  districts  and  schools.  "Disagree"  was  the  most  common  response  category  for 
each  of  the  six  items  and  over  60%  of  the  respondents  indicated  that  they  either  strongly  disagreed  or  disagreed  with 
the  six  statements  that  focused  on  ways  in  which  they  had  experienced  positive  local  impact  in  their  districts  and 
schools  as  a  result  of  educational  reform.  At  the  other  end  of  the  spectrum,  less  than  ten  percent  of  all  respondents 
strongly  agreed  that  educational  reform  had  a  positive  impact  on  any  of  the  six  areas  listed  above. 
There  were  significant  differences  in  the  pattern  of  responses  between  administrators  and  teachers.^  While  the 
responses  for  superintendents  and  principals  were  nearly  identical,  both  groups  were  quite  different  in  their  patterns 
of  responses  than  were  teachers.  The  administrators  (superintendents  and  principals)  were  more  likely  to  respond 


^  These  patterns  were  identified  through  the  use  of  exploratory  factor  analysis  and  confirmed  through  reliability 

analysis. 

^  Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "I  don't  know"  category  in  the  reporting  of  responses. 

^  Differences  among  the  three  groups  of  respondents  -  superintendents,  principals,  and  teachers  -  were  investigated 

through  the  use  of  independent  samples  t-tests. 
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favorably  to  each  of  the  six  items  than  were  teachers.  These  differences  were  large  and  highly  statistically 
significant.  In  fact,  the  average  score  for  each  of  the  six  items  is  almost  fifteen  percent  higher  per  item  for 
administrators  than  for  teachers. 


Communication  of  Goals 

Four  items  on  the  survey  focused  on  how  respondents  experienced  the  communication  of  goals  relating  to  education 
reform.  Table  2  presents  the  responses  for  these  items. 


Table  2  -  Response  Patterns  for  Items  Measuring  Expej 

rience  with  Communication  of  Goals 

Type  of  Experience 

Strongly* 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

State  policy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform 
Act  to  local  school  districts 

9.7% 

25.8% 

50.4% 

11.6% 

State  policy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform 
Act  to  the  general  public 

17.5% 

46.8% 

30.7% 

2.8% 

The  state's  priorities  under  Education  Reform  are 
clear 

6.7% 

23.7% 

50.0% 

18.7% 

1  understand  the  goals  of  the  Education  Reform 
Act 

0.6% 

3.3% 

41.7% 

54.2% 

Respondents  had  differing  views  on  how  well  the  goals  of  educational  reform  have  been  communicated  at  various 
levels.  In  general,  educational  professionals  at  the  district  and  school  levels  indicate  that  the  goals  and  priorities  of 
educational  reform  have  been  communicated  well  and  been  understood  in  districts  and  schools.  At  the  personal 
level,  respondents  feel  that  they  understand  the  goals  of  the  Education  Reform  Act;  with  approximately  96%  of 
respondents  agreeing  or  strongly  agreeing  that  they  understand  these  goals.  However,  these  same  individuals  are 
slightly  less  sure  that  the  goals  have  been  clearly  prioritized.  Just  over  two-thirds  of  the  respondents  agree  or 
strongly  agree  that  the  priorities  are  clear  and  almost  one-quarter  of  the  respondents  disagree  that  priorities  are  clear. 
Just  over  sixty  percent  of  the  respondents  indicated  that  they  have  been  well  informed  by  state  policy  makers  about 
the  goals  of  educational  reform.  In  contrast,  almost  65%  of  the  respondents  believe  that  state  policy-makers  have 
not  done  a  good  job  of  communicating  these  goals  to  the  general  public. 

As  was  the  case  for  the  first  group  of  items  measuring  experiences  relating  to  local  impact,  the  responses  to  the 
communication  of  goals  items  were  nearly  identical  for  administrators  as  a  whole;  but  significantly  different 
between  administrators  and  teachers.  Once  again,  administrators  were  about  15%  more  likely  than  were  teachers  to 
strongly  agree  or  agree  that  the  goals  and  priorities  of  educational  reform  have  been  well  communicated  by  state 
policy-makers  and  understood  at  local  levels. 


Local  Control 

The  next  set  of  items  measuring  perceived  experiences  with  education  reform  focus  on  the  ways  in  which  education 
reform  has  impacted  issues  associated  with  local  control.  Table  3  presents  a  summary  of  how  respondents  replied  to 
these  four  items. 


Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "I  don't  know"  category  in  the  reporting  of  responses. 
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Table  3  -  Response  Patterns  for  Items  Measuring  Experience  with  Issues  of  Local 

I  Control 

Type  of  Experience 

Strongly'' 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

Education  Reform  has  crowded  out  other 
initiatives  I  think  are  more  important 

4.8% 

43.8% 

29.1% 

16.9% 

Under  Education  Reform,  the  state  has  too  much 
power 

3.4% 

29.6% 

34.8% 

25.9% 

The  state  has  reasonable  expectations  regarding 
the  role  of  districts  in  implementing  the  Education 
Reform  Act 

12.0% 

54.3% 

20.3% 

9.2% 

The  state  has  reasonable  expectations  regarding 
the  role  of  schools  in  implementing  the  Education 
Reform  Act 

10.1% 

51.0% 

24.3% 

9.7% 

In  general,  respondents  indicated  that  they  have  not  had  favorable  impressions  of  the  ways  in  which  education 
reform  has  impacted  local  control  of  districts  and  schools.  Approximately  60%  of  the  respondents  agreed  or 
strongly  agreed  that  the  state  has  too  much  power  under  the  auspices  of  the  Education  Reform  Act.  Additionally, 
over  60%i  of  respondents  indicated  that  they  thought  that  the  state  has  had  unreasonable  expectations  regarding  the 
roles  of  districts  and  schools  in  implementing  education  reform.  It  is  interesting  to  note  that  respondents  were  fairly 
evenly  divided  as  to  whether  education  reform  has  crowded  out  other  important  initiatives. 

Like  the  previous  two  groups  of  experiential  items,  administrators  were  quite  homogenous  in  their  responses  to  these 
items  and  generally  had  a  more  favorable  impression  of  how  education  reform  has  impacted  local  control  than  did 
teachers. 


Use  of  Data 

Three  items  in  this  section  of  the  survey  focused  on  the  ways  in  which  the  state  collects  and  uses  data.  Table  4 
summarizes  the  responses  to  the  three  items  that  focused  on  respondents'  experience  with  the  collection  and  use  of 
data  under  education  reform. 


Table  4  -  Response  Patterns  for  Items  Measuring  Experience  with  Use  of  Data 


Type  of  Experience 

Strongly'" 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

The  Department  of  Education  collects  appropriate 
data  from  schools 

11.2% 

38.5% 

38.1% 

3.6% 

The  Department  of  Education  uses  these  data 
appropriately 

16.1% 

39.4% 

20.4% 

1.8% 

The  data  my  district  gets  from  the  Department  of 
Education  are  helpful  in  our  efforts  to  improve  the 
quality  of  education 

11.5% 

34.6% 

38.0% 

3.6% 

Responses  indicate  that  district  and  school  personnel  tend  to  disagree  that  the  Department  of  Education  is 
appropriately  collecting  and  using  data  in  ways  that  are  helpful  to  districts  and  schools.  However,  most  of  the 


'  Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "I  don't  know"  category  in  the  reporting  of  responses. 
'°  Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "1  don't  know"  category  in  the  reporting  of  responses. 
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responses  fall  in  the  middle  range  of  response  categories,  so  it  does  not  appear  that  there  are  strong  concerns  in  this 
area.  The  first  two  items  were  not  asked  of  teachers,  but  the  third  item,  measuring  how  helpful  data  from 
Department  of  Education  are  for  local  efforts  to  improve  the  quality  of  education,  is  viewed  much  less  favorably  by 
teachers  than  by  administrators.  Administrators  were  20%  more  likely  to  respond  more  favorably  to  this  item  than 
were  teachers.  This  may  indicate  a  difference  in  how  much  access  teachers  have  to  data  that  is  sent  from  the 
Department  of  Education  to  the  districts. 


State  Personnel 

The  remaining  questions  in  this  section  on  experiences  with  education  reform  focus  on  how  the  respondents  reported 
their  perceptions  of  their  interactions  with  state  personnel.  Table  5  presents  the  summary  of  the  responses  to  the 
three  questions  that  focus  on  the  human  resource  capacity  of  the  Department  of  Education  as  it  relates  to  the 
implementation  of  educational  reform. 

Table  5  -  Response  Patterns  for  Items  Measuring  Experience  with  State  Personnel 


Type  of  Experience 

Strongly" 
Disagree 

Disagree 

Agree 

Strongly 
Agree 

Department  of  Education  staff  members  have  the 
qualifications  and  experiences  they  need  to 
provide  useful  support  to  schools  and  districts 

21.4% 

27.8% 

17.5% 

0.6% 

The  Department  of  Education  seems  to  have 
enough  staff  to  carry  out  its  responsibilities 

28.2% 

30.7% 

12.8% 

3.9% 

When  I  or  my  staff  have  a  question  about 
Education  Reform,  the  Department  of  Education  is 
hebfiil 

12.4% 

30.3% 

35.1% 

3.9% 

The  majority  of  respondents  who  answered  these  items  had  an  unfavorable  impression  about  the  qualifications  and 
experiences  of  Department  of  Education  personnel.  The  respondents  were  even  less  likely  to  indicate  that  they 
believed  that  the  Department  of  Education  has  enough  personnel  to  carry  out  its  responsibilities.  Clearly, 
administrators  and  teachers  feel  that  the  Department  of  Education  is  under-prepared  and  under-staffed  to  be  able  to 
effectively  implement  education  reform.  The  responses  were  less  critical  of  the  helpfiilness  of  the  Department  of 
Education  when  answering  questions  about  education  reform.  However,  less  than  four  percent  of  respondents 
indicated  that  they  strongly  agreed  with  any  of  these  thee  statements;  indicating  that  satisfaction  with  the  human 
resource  of  Department  of  Education  to  be  moderate  at  best.  It  is  also  worth  noting  that  more  individuals  responded 
with  an  "I  don't  know"  to  these  three  items  than  to  any  of  the  other  twenty  experience-oriented  items.  Between  20%> 
and  34%  of  respondents  indicated  that  they  didn't  know  how  to  respond  to  any  one  of  these  three  items.  This  may 
indicate  that  many  district  and  school-level  personnel  have  not  had  enough  contact  with  the  Department  of 
Education  to  adequately  judge  the  human  resource  capacity  of  that  department. 

Teachers  were  much  more  likely  than  administrators  to  indicate  that  they  did  not  know  how  to  answer  these 
questions.  However,  administrators  were  almost  twice  as  likely  as  teachers  to  give  a  favorable  response  to  these 
items .  Additionally,  there  was  a  statistically  significant  difference  between  the  average  response  of  superintendents 
and  to  the  average  response  of  principals  to  the  question  about  the  experiences  and  qualifications  of  Department  of 
Education  personnel.  Superintendents  were  more  likely  to  have  a  favorable  impression  of  the  experience  and 
qualifications  of  Department  of  Education  personnel. 


' '  Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "I  don't  know"  category  in  the  reporting  of  responses. 
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Support  for  Education  Reform 

Eight  items  on  the  survey  focused  on  the  extent  to  which  respondents  felt  that  the  state  provided  support  for  the 
implementation  of  a  variety  of  education  reform  related  activities.  Table  6  summarizes  the  responses  to  the 
questions  about  support  for  education  reform. 


Table  6  -  Summary  of  Responses  to  Support  for  Education  Reform  Items 


Type  of  Support 

Poor^^ 

Fair 

Good 

Excellent 

Aligning  curriculum  with  the  state's  frameworks 

20.5% 

27.1% 

30.9% 

5.9% 

Providing  support  for  students  who  score  poorly 
on  MCAS 

29.2% 

37.5% 

24.2% 

5.6% 

Interpreting  MCAS  results 

21.1% 

35.6% 

30.6% 

9.2% 

Developing  human  resource  policies  to  improve 
staff  quality 

43.0% 

30.9% 

9.6% 

1.7% 

Professional  development  for  teachers 

23.1% 

38.1% 

30.3% 

4.2% 

Professional  development  for  administrators 

32.5% 

31.9% 

18.9% 

2.8% 

Improving  schools'  overall  performance 

27.5% 

45.7% 

20.2% 

1.4% 

Meeting  the  needs  of  special  populations 

56.5% 

28.1% 

10.0% 

1.9% 

While  the  perceptions  about  experiences  were  quite  mixed,  the  respondents  were  generally  unfavorable  in  their 
assessment  of  state  support  in  these  eight  key  areas.  Levels  of  support  were  rated  as  "poor"  at  least  20%  across  all 
eight  items.  Support  for  meeting  the  needs  of  special  populations  appears  to  be  the  area  of  biggest  concern,  with 
over  55%  of  respondents  rating  the  support  as  poor  and  less  than  2%  rating  support  in  this  area  as  excellent.  There 
also  appears  to  be  widespread  concern  about  support  for  developing  human  resource  policies  to  improve  staff 
quality,  as  43%  of  respondents  rated  this  area  poorly  and  less  than  2%  rated  support  as  being  excellent.  Support  for 
interpreting  MCAS  results  was  rated  most  highly  among  these  eight  indicators,  but  even  this  indicator  was  ranked  as 
excellent  less  than  ten  percent  of  the  time  and  as  poor  over  20%  of  the  time  by  respondents. 

Perhaps  the  most  insightful  analysis  into  the  extent  to  which  district  and  school  level  educators  are  concerned  about 
support  for  special  needs  populations  and  the  development  of  human  resource  policies  to  improve  staff  quality 
comes  from  an  examination  in  the  differences  between  the  average  response  from  teachers  to  these  support  items 
and  the  average  response  from  the  administrative  perspective.  Administrators  reported  significantly  higher  ratings 
of  support  than  did  teachers  on  all  of  the  items  measuring  aspects  of  support,  except  for  two  items  -  those  that  dealt 
with  special  needs  populations  and  the  development  of  human  resource  policies  to  improve  staff  quality.  Both 
administrators  and  teachers  ranked  these  areas  as  being  supported  at  similarly  low  levels  by  the  state. 

There  were  also  some  notable  patterns  of  open-ended  responses  related  to  issues  of  support.  In  particular,  three 
issues  were  identified  that  relate  to  issues  of  support.  These  include: 

•  Lack  of  sufficient  quantities  of  human  resources  in  the  Department  of  Education 

•  Lack  of  qualified  personnel  in  the  Department  of  Education 

•  Too  many  demands  placed  on  the  Department  of  Education 

The  most  prevalent  patterns  of  issues  related  to  support  identified  in  the  open-ended  responses  focused  on  the 
Department  of  Education's  need  for  a  greater  quantity  of  personnel.  Some  of  the  comments  in  this  area  included: 

•  It  appears  more  human  resources  are  needed  at  the  DOE  to  fully  assist  local  districts. 

•  Their  (the  Department  of  Education)  staff  has  actually  been  reduced  over  the  last  several  years. 


'^  Percentages  do  not  add  up  to  100  due  to  the  omission  of  an  "1  don'tknow"  category  in  the  reporting  of  responses. 
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•  The  DoE  staff  is  under- funded  and  under-staffed 

•  Insufficient  personnel  to  answer  questions. 

•  Although  the  Doe  staff  tries  to  be  helpful,  they  are  short-staffed. 

•  It  seems  that  they  are  always  under-staffed. 

•  Personnel  in  the  department  are  over-worked.  For  example,  individuals  have  been  given  responsibility  for 
for  school  building  assistance  without  being  relieved  of  other  duties. 

•  Human  resources  are  becoming  an  on-going  problem  as  the  scarcity  of  personnel  in  grants  management  and 
certification  has  impacted  regular  district-wide  inquiries  and  decisions. 

•  They  need  more  personnel  to  advise  and  consult  in  the  area  of  professional  development  for  classroom 
teachers  and  guidance  counselors. 

Many  respondents  also  indicated  that  they  thought  that  many  staff  members  at  the  Department  of  Education  are 
under-qualified  for  their  positions  and/or  lack  experience  that  helps  them  understand  what  really  happens  in  districts 
and  schools.  Examples  of  comments  related  to  this  theme  include: 

•  Too  many  DOE  staff  have  never  worked  in  schools.  They  lack  contextual  awareness  needed  to  provide 
support  for  change. 

•  Many  of  the  employees  have  no  clue  about  day-to-day  school  operations  including  several  on  the  Board  of 
Ed... 

More  DoE  staff  need  to  have  experience  working  in  schools  as  public  teachers  or  administrators. 

No  -  more  expertise  is  needed.  Pay  experienced  teachers  to  do  the  job. 

I  don't  know!  I  feel  many  people  aren't  qualified  for  the  positions  they  are  in. 

•  Personnel  do  not  have  tech  or  financial  resources  and  it  is  my  impression  that  many  lack  the  educational 
experience  necessary  to  recognize  feasible  solutions  to  situations  which  arise. 

The  Department  of  Education  also  has  too  many  demands  placed  on  it  with  too  few  resources  provided  to  assist 
them  in  meeting  of  all  of  these  challenges.  Some  respondents  note: 

•  At  present,  the  mandates  of  Ed  Reform  are  too  broad  for  DOE  to  address  every  area  and  every  school 
adequately. 

•  DoE  is  over-whelmed  by  all  they  have  to  do. 

•  Ed.  reform  has  created  too  much  to  do  at  all  levels  -  including  the  state. 

State  Versus  Local  Responsibility  for  Education  Reform 

One  of  the  key  issues  in  determining  the  state's  capacity  for  implementing  education  reform  is  the  extent  to  which 
the  state  and  local  levels  of  the  education  are  primarily  or  jointly  responsible  for  implementing  education  reform. 
Therefore,  respondents  were  asked  to  indicate  whether  the  following  twelve  tasks     of  education  reform  were 
primarily  the  responsibility  of  the  state,  primarily  the  responsibility  of  local  districts  and  schools,  or  a  shared 
responsibility  between  state  and  local  authorities.  The  results  from  these  items  can  be  found  below  in  Table  7. 


'^  These  twelve  tasks  were  identified  by  the  research  team  as  being  areas  that  represent  key  responsibilities  of 
education  reform. 
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Table  7  -  Summary  of  Responses  to  State  Versus  Local  Responsibility  Items 


Task 

Primarily 

State 

Responsibility 

Shared 
Responsibility 

Primarily 

Local 

Responsibility 

Developing  and  implementing  curriculum  standards 

12.1 

76.1 

11.5 

Developing  and  administering  a  student  assessment  system 

10.7 

72.8 

16.3 

Developing  an  accountability  system  for  school  and  district 
performance 

14.6 

72.5 

12.6 

Supporting  local  governance  and  management  of  education 
reform 

14.2 

63.2 

19.7 

Developing  and  administering  processes  for  certifying, 
evaluating,  and  dismissing  school  personnel 

14.8 

52.6 

31.8 

Providing  and/or  guiding  professional  development 

2.3 

58.6 

38.5 

Allocating  funding  in  a  way  that  is  adequate  and  equitable 

42.6 

47.2 

9.7 

Facilitating  inter-district  transfer 

19.7 

27.9 

37.1 

Supervising  charter  schools 

58.5 

22.2 

8.5 

Ensuring  readiness  to  learn  through  early  childhood  and  literacy 
programs 

11.0 

70.9 

15.8 

Making  state-level  policy  and  planning  decisions  in 
coordination  with  key  actors 

31.6 

62.6 

1.4 

Collecting  and  using  information  to  improve  the  performance  of 
the  state  educational  system 

30.4 

65.6 

2.8 

Respondents  were  more  likely  to  perceive  that  ten  of  the  twelve  tasks  are  a  shared  responsibility,  rather  than 
primarily  either  a  state  or  local  responsibility.  One  of  the  tasks,  supervising  charter  schools,  was  viewed  by  a 
majority  of  the  respondents  (58.5%)  as  being  primarily  a  state  responsibility.  In  contrast,  more  respondents 
indicated  that  facilitating  inter-district  transfer  was  primarily  a  local  responsibility.  Although  adequate  and  equitable 
allocation  of  funding  is  seen  as  primarily  a  shared  responsibility  (47.2%),  almost  as  many  respondents  (42.6%) 
viewed  this  task  as  being  primarily  a  state  responsibility. 

Providing  and  guiding  professional  development  was  one  area  that  respondents  clearly  viewed  as  not  being  a  state 
responsibility,  with  only  2.3%  of  respondents  indicating  that  this  was  a  state  responsibility  from  their  perspective. 
On  the  other  hand,  respondents  did  not  view  supervision  of  charter  schools  (8.5%)),  making  state-level  policy  and 
planning  decisions  (1.4%)),  or  collecting  and  using  information  to  improve  the  performance  of  the  state  educational 
system  (2.8%)  as  being  local  responsibilities. 

There  were  no  statistically  significant  differences  between  superintendents  and  principals  on  any  of  the 
responsibility  items.  There  were  significant  differences  between  administrators  and  teachers  on  two  of  the  items  - 
developing  an  accoimtability  system  for  school  and  district  performance  and  developing  and  administering  processes 
for  certifying,  evaluating,  and  dismissing  school  personnel.  In  both  cases,  teachers  were  more  likely  to  report  that 
these  tasks  were  primarily  local  responsibilities  while  administrators  were  more  likely  to  report  that  they  were 
shared  responsibilities.  Interestingly,  superintendents  and  principals  also  differed  significantly  on  the  issue  of 
responsibility  for  developing  and  administering  processes  for  certifying,  evaluating,  and  dismissing  school 
personnel.  Both  administrative  groups  most  frequently  reported  this  task  as  a  shared  responsibility,  but  of  those  who 
did  not  view  this  as  a  shared  responsibility,  superintendents  were  more  likely  to  view  this  as  a  state  responsibility 
while  principals  who  didn't  view  these  processes  as  a  shared  responsibility  were  more  likely  to  view  it  as  a  local 
responsibility. 
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Flow  of  Information 

The  survey  also  asked  respondents  to  provide  some  information  regarding  the  flow  of  information  to  professionals 
in  districts  and  schools.  Four  types  of  questions  were  included  on  the  survey  that  focused  on  the  flow  of 
information.  Respondents  were  asked  how  frequently  they  contacted  the  Department  of  Education,  how  easy  it  was 
to  get  information  from  the  Department  of  Education,  how  they  received  information  about  education  reform,  and 
how  they  contacted  individuals  at  the  Department  of  Education.  The  responses  to  these  questions  are  summarized 
below. 

Table  8  -  Summary  of  Responses  Regarding  Frequency  of  Contact 


How  frequently  do  you  contact 
the  Deparlinent  of  Education? 

Never 

3.7% 

Once  Per  Year 

7.1% 

Several  Times  Per  Year 

60.9% 

Once  Per  Month 

15.6% 

Once  Per  Week 

10.8% 

More  Than  Once  Per  Week 

2.2% 

Table  8  indicates  that  most  (over  60%)  respondents'"*  contact  the  department  several  times  per  year,  with  many 
fewer  respondents  indicating  that  they  contact  the  Department  of  Education  less  frequently.    Most  of  the  remaining 
respondents  indicating  that  they  contact  the  Department  of  Education  at  least  once  per  month,  if  not  more  frequently. 
In  an  analysis  of  differences  between  the  two  administrative  groups,  superintendents  were  significantly  more  likely 
to  contact  the  Department  of  Education  than  were  principals. 


Table  9  -  Summary  of  Responses  Regarding  Ease  of  Obtaining  Information 


How  easy  is  it  to  get  information 
from  the  Department  of  Education? 

Very  Easy 

5.8% 

Easy 

26.3% 

Difficult 

57.0% 

Very  Difficult 

10.9% 

Respondents  were  also  asked  to  indicate  ways  in  which  they  received  information  about  education  reform.  Table  9 
shows  that  the  majority  (57%)  of  respondents  indicated  that  it  was  difficult  to  get  information  from  the  Department 
of  Education.  There  were  no  differences  between  superintendents  and  principals  on  this  item,  but  teachers  were 
more  likely  to  indicate  that  they  found  it  difficult  to  obtain  information  from  the  Department  of  Education. 


14 


Only  superintendents  and  principals  were  asked  this  question,  as  teachers  were  unlikely  to  directly  contact  the 
Department  of  Education  with  any  frequency. 
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Table  10  -  Summary  of  Responses  Regarding  Sources  of  Information  About  Education 
Reform 


How  do  you  get  information  about  education  reform'^? 

Documents  mailed  directly  to  me  by  the  Department  of  Education 

87.9% 

The  Department  of  Education  web  site 

69.0% 

The  media 

56.6% 

Via  a  union  or  professional  association 

55.5% 

Documents  mailed  by  the  Department  of  Education  to  districts  and  distributed  in  schools 

49.7% 

Documents  mailed  directly  to  me  by  other  state  agencies  or  offices 

29.6% 

Table  10  indicates  that  most  respondents  (approximately  88%)  obtained  information  about  education  reform  directly 
from  the  Department  of  Education.  However,  almost  every  respondent  used  multiple  sources  of  information  to  learn 
about  education  reform. 

Superintendents  were  more  likely  to  get  documents  from  other  state  agencies  or  offices  and  were  more  likely  to  get 
information  from  the  Department  of  Education's  web  site  than  were  either  principals  or  teachers.  Teachers  were 
less  likely  than  either  group  of  administrators  to  get  information  directly  from  the  Department  of  Education  or  from 
the  Department's  web  site.  However,  teachers  were  more  likely  to  use  the  media,  union  and  professional 
associations,  and  documents  passed  on  at  the  local  level  from  the  state  as  sources  of  information  than  were 
superintendents  and  teachers. 


Table  11  -  Summary  of  Responses  Indicating  How  Respondents  Most  Frequently  Contact 
the  Department  of  Education 


How  do  you  (or  your  staff)  most  frequently  contact  someone  at  the  Department  of  Education'^? 

Item 

Teachers 

Admins. 

All 

I  directly  contact  a  specific  department  or  program  area 

19.5% 

55.2% 

47.5% 

I  use  the  website  to  find  the  appropriate  contact  person  and  I  call  him/her 

18.2% 

40.1% 

35.3% 

I  directly  contact  someone  I  already  know 

6.5% 

41.9% 

34.2% 

I  use  the  switchboard  operator  to  direct  me  to  the  appropriate  contact  person 

29.9% 

23.5% 

24.9% 

I  use  the  website  to  find  the  appropriate  contact  person  and  I  send  him/her  an 
email  message 

18.2% 

40.1% 

18.1% 

I  do  not  contact  the  Department  of  Education 

44.2% 

7.2% 

15.3% 

Flow  of  information  was  also  investigated  in  the  other  direction  as  respondents  were  asked  how  they  (or  their  staff) 
most  frequently  contacted  the  Department  of  Education.  Table  1 1  indicates  that  most  respondents  (47.5%)  contact 
the  Department  of  Education  by  directly  contacting  a  specific  department  or  program  area.  Of  those  individuals 
who  reported  contacting  the  Department  of  Education,  respondents  were  least  likely  (18.1%)  to  report  using  the 
Department's  web  page  and  following  up  with  an  email,  although  using  the  web  page  to  get  a  phone  number  was  the 
second  most  popular  (35.3%)  means  of  contacting  the  Department  of  Education.  Just  over  15%  of  respondents 
indicated  that  they  do  not  contact  the  Department  of  Education. 

Administrators,  superintendents  and  principals,  were  more  likely  to  report  higher  frequencies  of  directly  contacting  a 
specific  department  or  program  area,  using  the  web  page  to  get  phone  numbers,  and  using  the  switchboard  operator 
than  were  teachers.  However,  teachers  and  principals  were  more  likely  to  contact  someone  they  knew  than  were 
than  were  superintendents.    It  is  also  interesting  to  note  that  thee  were  no  significant  differences  among  the  three 
groups  with  regard  to  the  item  indicating  no  contact  with  the  Department  of  Education. 


'^  Respondents  were  allowed  to  mark  more  than  one  response  in  order  to  identify  all  sources  of  information  used  by 

any  one  respondent. 

'^  Some  respondents  checked  more  than  one  response  for  this  item,  resulting  in  percentages  that  exceed  100%. 
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There  were  also  a  number  of  comments  from  the  open-ended  questions  that  related  to  flow  of  information.  More 
specifically,  many  respondents  commented  on  how  slow  the  Department  of  Education  is  at  remming  calls  and 
answering  inquiries. 

I  have  on  several  occasions  called  for  assistance  regarding  state  reports  only  to  not  have  my  calls  returned. 

The  web  page  is  very  informative  and  well  done,  however  it  is  almost  impossible  to  get  someone  on  the 
phone  to  answer  a  question. 

•  My  only  contact  with  DOE  personnel  comes  when  I  call  with  a  question.  Most  times,  people  with  whom  I 
speak  are  helpful,  but  reluctant  to  give  definitive  answers. 

•  It's  nearly  impossible  to  contact  the  DOE  departments  regarding  certification  and  the  frameworks. 

•  Massive  change.  Inefficient  system,  and  follow  through.  No  help  on  the  phone  (20  messages)  when 

MC  AS  did  not  arrive.  Staff  lacks  professionalism.  Certification  department  a  total  embarrassment.  Can't 
seem  to  make  deadlines  -  looks  good  on  paper  but  not  in  reality. 

•  They  need  to  be  more  timely  in  providing  answers,  answer  phones,  etc. 

Others  noted  that  the  Department  of  Education  has  greatly  improved  communication  through  better  use  of 
technology  and  information  systems.  Sample  comments  included: 

•  However,  they  have  improved  with  communication  in  previous  years. 

•  I  feel  that  the  DOE  does  an  excellent  job  of  distributing  data  and  collecting  data. 

•  The  web  page  is  help  fill  and  getting  better  all  of  the  time. 


Relationships  Among  Key  Indicators 

In  addition  to  the  descriptive  analysis  provided  above,  relationships  among  eight  key  indicators  were  also 
investigated.  In  particular,  the  relationships  among  the  five  experience  factors  (composite  items  representing  the 
experience  items  groupings  for  local  impact,  communication  of  goals,  local  control,  use  of  data,  and  state 
personnel),  a  composite  scale  of  the  support  items  (labeled  "support"),  the  fi^equency  of  contact  variable,  and  the 
variable  measuring  how  easy  it  is  to  get  information. 

The  frequency  of  contact  measure  had  very  few  statistically  significant  relationships  with  the  other  seven  variables. 
The  more  individuals  reported  interacting  with  the  Department  of  Education  the  more  likely  they  were  to  report 
positive  local  impact  from  education  reform  effort  and  the  more  likely  they  were  to  report  having  positive 
experiences  with  state  personnel.  However,  these  were  the  weakest  of  the  statistically  significant  relationships 
among  any  of  the  variables  included  in  this  particular  analysis. 

All  of  the  relationships  among  the  remaining  seven  variables  were  positive  and  statistically  significant.  This  pattern 
of  findings  indicates  that  individuals  who  have  positive  experiences  in  one  area  of  education  reform  are  more  likely 
to  perceive  positive  experiences  in  other  areas,  are  more  likely  to  report  high  levels  of  support  from  the  state,  and  are 
more  likely  to  believe  that  information  regarding  education  reform  is  easy  to  obtain.  Figure  1  shows  this  pattern  of 
relationships  among  these  key  indicators. 
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Figure  1  -  Significant  Relationships  Among  Key  Indicators 


Experience 
Factors 

•  Local  Impact 

•  Communication 

of  Goals 

•  Local  Control 

•  Use  of  Data 

•  State  Personnel 


Implementation  of  Education  Reform  Components 

Respondents  were  provided  the  opportunity  to  provide  open-ended  responses  regarding  the  components  of  education 
reform  that  have  not  yet  been  implemented.  The  most  common  responses  included: 

•  Adequate  funding 

•  Fully  integrated  curriculum  frameworks 

•  Professional  development 

•  Early  childhood  and  literacy  programs 

•  Comprehensive  assessment 

•  Nothing 

Funding  appears  to  remain  as  a  major  unresolved  issue  for  superintendents,  principals,  and  teachers  alike.  The 
comments  below  reflect  a  wide  range  of  ways  in  which  many  local  educators  feel  that  adequate  and  equitable 
funding  has  not  yet  been  fully  implemented  or  realized  throughout  the  Commonwealth  as  a  result  of  education 
reform. 

•  Funding  -  $  -  need  for  more  $.  When  state  contribution  to  city  or  town  decreases  it  is  difficult  to  assist 
students. 

•  Money  for  materials!  Difficult  to  get  grant  money  for  the  programs  you  want. 

•  Greater  financial  support  for  schools  in  terms  of  providing  them  with  resources  to  be  able  to  meet  the 
standards. 


I  am  not  convinced  that  appropriate  financial  resources  are  available  to  poorer  students  in  poorer  schools. 

Funding  sources  for  the  implementation  of  Ed  Reform  has  been  insufficient  -  especially  related  to 
curriculum  revisions  and  MCAS  administration. 


There  were  also  concerns  that  curriculum  frameworks  have  not  been  fully  developed  or  implemented,  as  some  of  the 
comments  below  demonstrate. 
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• 


Common  core  of  learning  -  world  language  requirement  have  not  been  implemented.  Our  school  district 
has  embraced  this  as  a  direction  -  yet  the  state  gives  little  or  no  acknowledgment  of  this  fact. 

Clearly  established  curriculum  frameworks  that  are  used  for  a  consistent  period  of  time. 


•     Getting  those  frameworks  in  place  and  keeping  them  there! 


• 


The  implementation  of  MCAS  testing  was  done  much  too  rapidly.  There  was  not  enough  time  or  money 
given  to  align  curriculum  with  frameworks. 


Professional  development  also  remains  a  concern  as  an  area  of  education  reform  that  has  not  been  fiilly 
implemented. 

•  Ed.  reform  has  had  virtually  no  impact  on  professional  development. 

•  The  state  keeps  changing  what  we  are  expected  to  do,  but  never  provides  us  with  training  about  how  to  do 
the  new  things. 

•  It's  unclear  what  is  supposed  to  happen  with  professional  development. 

•  The  new  system  focuses  more  on  punishing  teachers  rather  than  on  improving  teachers. 

•  There  is  less  time  for  professional  development  because  of  all  of  the  other  things  educators  have  to  do  in 
order  to  comply  with  ever-changing  state  mandates. 

Some  respondents  also  indicated  the  early  childhood  education  and  literacy  had  not  yet  been  adequately  addressed 
or  implemented  as  a  part  of  education  reform.  Some  respondents  noted  the  following  observations  in  this  area. 

•  The  whole  early  childhood  thing  has  been  left  out  so  far.  I  hope  that  changes  soon. 

•  Until  we  may  more  attention  to  early  childhood  issues,  especially  in  urban  and  other  disadvantaged  areas, 
education  reform  is  doomed  to  fail. 

Comprehensive  assessment  was  widely  reported  to  be  an  unimplemented  part  of  education  reform  according  to 
several  respondents. 

•  Multiple  types  of  assessment  processes  are  still  needed. 

•  Assessment  in  Science  and  Social  Studies  have  not  been  included  in  the  graduation  requirement.  Goals  to 
include  foreign  language  in  testing  and  graduation  requirements  long  ago. 

•  Multiple  ways  to  earn  graduation  (are  needed). 

•  Multiple  assessments  =  Good  education. 

Finally,  despite  the  fact  that  the  majority  of  comments  tended  to  be  negative  and  focus  on  the  incomplete  aspects  of 
the  implementation  of  education  reform,  there  were  also  large  number  of  individuals  who  indicated  that  nothing  else 
needed  to  be  implemented  or  that  education  reform  was  working  and  having  positive  effects.  Some  of  the  comments 
to  this  effect  include  the  following  quotes  taken  fi-om  the  open-ended  questions  on  the  survey. 

•  As  a  result  of  the  Education  Reform,  I  find  instructors  practices  are  now  systematized,  and  our  instructional 
programs  are  data  driven,  and  our  professional  development  is  developed  around  specific  instructional 
needs. 

•  Educational  Reform  has  had  some  positive  influences  regarding  staffing,  professional  development,  etc. 

•  Overall  Ed  Reform  has  had  a  positive  impact  on  my  school.  Professional  development  opportunities  for 
staff  have  increased  significantly  and  school  council  has  enabled  us  to  set  clear  goals  for  our  school. 

•  I  support  it  because  all  students  will  be  expected  to  meet  high  standards. 
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Recommendations  for  Change 

Respondents  were  also  asked  in  an  open-ended  question  about  their  recommendations  for  change.  While  several 
themes  emerged  in  response  to  this  question,  many  respondents  emphasized  the  need  for  more  constructive 
relationships  between  the  states  and  local  districts.  This  sentiment  was  articulated  in  a  variety  of  ways  including  the 
following  statements. 

•  Make  sure  everybody  at  the  DOE  is  on  the  same  page.  We  receive  too  much  contradictory  information. 
Make  sure  the  DOE  really  wants  us  to  do  what  it  is  about  to  require. 

•  Please  listen  carefully  to  the  educators  in  the  field. 

•  More  professional  assistance  for  schools  and  less  bureaucratic  red  tape. 

•  I  feel  that  the  DOE  does  an  excellent  job  of  distributing  data  and  collecting  data.  I  would  like  to  see  the 
DOE  be  more  collaborative  with  schools  and  districts  in  developing  instructional  plans  that  improve 
student  learning,  i.e.  a  resource  team  that  could  work  with  my  school  on  implementing  our  school 
improvement  plan  goals. 

•  Come  and  talk  to  us ! 

As  a  related  suggestion,  many  respondents  indicated  the  need  for  more  visits  to  local  schools  by  the  Department  of 
Education  and  other  state  officials.  Sample  comments  in  this  area  include: 

•  DOE  employees  need  to  get  out  into  the  districts  and  talk  to  and  listen  to  the  teachers,  principals,  etc. 

•  Have  people  from  DOE  -  come  out  to  model  in  districts  -  do  prof.  dev.  -  observe  in  schools  for  long 
periods  of  time. 

•  I  make  informal  visits  to  each  classroom  in  my  building  everyday.  In  this  way,  I  get  to  know  my  students 
and  staff.  How  often  do  DOE  staff  visit  the  schools  whose  fates  they  hold  in  their  hands? 

Other  Issues 

Several  issues  arose  in  other  opportunities  for  open-ended  responses.  Most  frequently,  a  number  of  individuals 
reported  that  the  state,  under  the  auspices  of  education  reform,  has  made  too  many  contradictory  demands  on 
schools. 

•  Yes,  the  curriculum  frameworks  keep  changing  radically.  As  soon  as  we  get  things  in  a  line,  everything 

shifts  again,  ie:  Sci  &  tech.  -  we  setup  a  sequence  of  integrated  science  (9)  and  biology  (10)  and  intro  to 
technology  (9).  Now  the  science  curriculum  is  being  altered  which  would  have  allowed  our  previous 
curriculum  to  have  remained. 

•  As  is  the  case  with  any  bureaucracy,  there  sometimes  are  mixed  messages.  I  frequently  feel  that  the  DOE 
staff  have  completely  lost  touch  with  day  to  day  operations  of  a  public  school. 

•  Changing  policies  on  frameworks/  first  round  of  MCAS  was  going  to  be  base  line?  Recertification  went 
from  a  positive  exercise  to  a  punitive  experience.  Teacher  bashing  has  had  negative  effects. 

•  Yes  -  they  want  us  to  improve  instruction,  use  technology,  teach  students  to  be  technologically  savvy,  have 
students  work  in  teams.  MCAS  runs  counter  to  all  of  this. 

•  Absolutely!  They  keep  changing  the  frameworks  and  the  testing  schedule! 
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Conclusion 


Educators  at  various  levels  -  district  superintendents,  school  principals,  and  classroom  teachers  -  are  heavily 
invested  in  and  strongly  influenced  by  education  reform.  The  findings  from  this  survey  suggest  that  many  educators 
at  the  local  level  are  frustrated  by  a  variety  of  issues  associated  with  the  implementation  of  education  reform.  On 
the  other  hand,  there  are  also  many  educators  who  report  progress  and  satisfaction  with  education  reform  and  its 
implementation. 

In  general,  administrators  appear  to  be  better  informed  and  more  satisfied  with  the  levels  of  implementation  of 
education  reform  than  teachers.  It  is  not  clear  from  the  data  collected  from  this  study  what  the  reasons  for  the 
information  and  satisfaction  gap  between  adminisfrators  and  teachers.  This  is  an  issue  that  merits  further  study. 

Clearly,  communication  and  involvement  are  important  to  local  educators.  The  state  may  want  to  consider  ways  in 
which  communication  and  involvement  in  improving  the  state's  capacity  for  implementing  education  reform  can  be 
improved.  The  examination  of  relationships  among  key  variables  from  this  survey  suggests  that  there  is  a  positive 
relationship  between  levels  of  perceived  support,  the  ease  at  which  information  can  be  obtained  and  educators 
having  more  positive  experiences  with  education  reform. 

The  educators  who  responded  to  this  survey  also  view  the  implementation  of  the  tasks  associated  with  education 
reform  as  primarily  a  shared  responsibility  between  the  state  and  local  districts  and  schools.  Many  of  the  responses 
to  open-ended  questions  also  suggest  that  finding  ways  to  improve  communication  and  participation  across  state  and 
local  boundaries  has  the  potential  to  improve  the  state's  capacity  for  education  reform. 
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Appendix  3 

State  vs.  Federal  FTE's  at  the  Massachusetts  Department  of  Education 


FISCAL  YR. 

FEDERAL  FTE 

STATE  FTE 

TOTAL 

1980 

367 

623 

990 

1981 

367 

605 

972 

1982 

349 

524 

873 

1983 

310 

516 

826 

1984 

303 

487 

790 

1985 

296 

286 

582 

1986 

299 

357 

656 

1987 

277 

350 

627 

1988 

255 

366 

621 

1989 

254 

354 

608 

1990 

254 

275 

529 

1991 

232 

184 

416 

1992 

212 

172 

384 

1993 

180 

145 

325 

1994 

200 

142 

342 

1995 

198 

148 

346 

1996 

193 

142 

335 

1997 

193 

145 

338 

1998 

191 

147 

338 

1999 

183 

216 

399 

2000 

171 

224 

395 

2001 

183 

222 

405 
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Appendix  4 

Interview  Protocols  and  Survey  Instruments 

Interview  Protocols 

1 .  Local/District  Personnel 

I'm  interviewing  you  as  part  of  a  study  of  the  state's  capacity  to  carry  out  its  responsibilities 
under  the  Education  Reform  Act.  The  team  carrying  out  the  study  is  based  at  the  Center  for 
Education  PoHcy  of  the  School  of  Education  at  the  University  of  Massachusetts,  Amherst. 

Our  goal  for  the  study  is  to  produce  positive,  constructive  recommendations  to  state  policy 
makers.  We  want  to  take  stock  of  various  offices'  responsibilities  for  Education  Reform,  how 
well-equipped  they  have  been  to  carry  out  these  responsibilities,  and  how  they  might  be  better 
supported. 

The  study  is  sponsored  by  the  Massachusetts  Education  Reform  Review  Commission.  The 
Commission  was  established  by  the  state  legislature  in  1 993  to  oversee  implementation  of  the 
Education  Reform  Act.  The  Commission's  primary  activity  is  funding  independent  research  on 
various  education  reform  related  topics. 

The  findings  from  our  study  will  be  made  public  later  this  year.  What  you  tell  me  today  may  be 
included  in  the  report,  but  will  not  be  attributed  to  you  by  name  or  by  position. 

I'd  Uke  to  make  a  tape  of  this  interview  so  that  we  have  an  accurate  record  of  what  was  said.  Is 
this  OK?  If  you  want  me  to  turn  off  the  tape  recorder  at  any  poiat,  please  say  so. 

Background  questions  - 

What  is  your  position? 

How  long  in  this  position? 

How  long  have  you  worked  for  (school  or  district)? 

Where  did  you  work  before  this  position? 


• 


• 


1.   I'd  like  to  start  with  some  questions  about  how  Education  Reform  has  affected  your 
work,  and  how  you've  worked  with  state  agencies. 

•  How  has  your  job  has  changed  as  a  result  of  MERA? 

•  How  effective  was  the  DoE  in  communicating  with  you  about  these  changes,  if  any? 

•  What  do  you  understand  to  be  the  state's  responsibilities  and  the  districts'  responsibilities  in 
implementing  the  Education  Reform  Act?  Do  you  think  this  balance  is  appropriate? 

(Prompt  for  feedback  about  different  types  of  responsibilities  -  technical  assistance, 
regulation,  finance,  and  research  and  data  collection.) 

•  How  has  Education  Reform  affected  your  district's  priorities? 
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•  Has  the  DoE  given  you  appropriate  help  doing  your  job?  If  so,  how?  If  not,  what  was 
lacking? 

•  How  often  do  you  or  others  you  know  in  your  district  or  school  interact  with  DoE? 

Prompt:  with  what  person  or  what  office  most  frequently? 

•  How  Would  you  characterize  those  interactions? 

•  Have  these  interactions  with  the  DoE  changed  since  the  implementation  of  MERA? 

2.  I'm  also  interested  in  knowing  how  information  gets  communicated  between  DoE  and 
local  districts. 

•  What  kinds  of  information,  opinions,  or  other  inputs  does  your  school  or  district  send  to 
DOE,  and  to  whom?  Are  there  others  you  think  you  should  be  sending?  t 

•  What  kinds  of  information,  guidance,  mandates,  or  other  output  does  DoE  give  to  your  1 
district?  From  whom  is  it  sent?  What  would  you  like  to  receive?  I 

•  Is  it  your  impression  that  this  process  is  efficient  and  effective?  \ 

•  Are  there  other  state  agencies  to  which  you  submit  information,  or  from  which  you  get  ! 

information  and/or  guidance?  ! 

i 

3.  Now,  I'd  like  to  ask  a  few  questions  about  your  perceptions  of  how  DOE  works.  \ 

•  What  do  you  see  as  the  goals  and  priorities  of  the  DoE?  Who  sets  them?  f 

•  How  have  these  goals  and  priorities  changed  over  time? 

•  To  what  degree  do  you  see  MERA  implementation  as  the  major  goal  and  number-one 
priority  of  the  department? 

4.  The  page  I'm  giving  you  summarizes  11  key  components  of  the  Education  Reform  Act  of 
1993.  Given  this  description  of  the  Act,  what  would  you  say  about  the  state's  capacity,  or 
ability,  to  implement  it?  What  has  the  state  done  well,  and  what  has  it  done  less  well?  (Ask 
respondents  to  specify  who  they  mean  when  they  refer  to  "the  state"— try  to  get  more  specific 
than  "DOE."  Also  prompt  if  necessary  about  different  types  of  responsibilities — ^technical 
assistance,  regulation,  finance,  data  collection  and  information.) 

Curriculum  standards 

Student  assessment  system 

Accountability  system  for  school  and  district  performance 

Local  govemance  and  management  of  schools  and  districts 

Processes  for  certifying,  evaluating,  and  dismissing  school  personnel 

Professional  development 

School  fmance/funding 

Choice  and  charter  schools 

Readiness  to  learn/early  childhood/literacy 

State-level  policy  and  planning 

Collecting  and  using  information  to  improve  system  performance  | 
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4.    Finally,  I'd  like  to  hear  your  recommendations  for  the  future  implementation  of 
Education  Reform. 

•  What  advice  would  you  give  to  MERRC  about  the  state's  capacity  to  implement  education 
reform? 

•  What  could  the  DoE  do  that  would  be  most  helpful  to  you  in  doing  your  job  more 
effectively? 

•  Are  there  other  states  or  models  of  managing  education  reform  that  you  find  interesting? 
Department  of  Education  Personnel 

Introductory  material  same  in  this  protocol  as  in  the  Local/District  Protocol 

Background  questions  - 

•  How  long  have  you  worked  at  DoE,  and  in  what  capacity? 

•  Where  did  you  work  before  DoE?  School-  or  district-based  experience? 

•  Why  did  you  come  to  DOE? 

1.  First,  I'd  like  to  get  a  sense  of  what  your  job  entails. 

•  Could  you  briefly  describe  your  job  for  me,  your  main  day-to-day  tasks? 

Prompt:  state  or  federal  program/mandate/funds,  or  both?  Other  grants? 

•  Do  you  work  more  with  people  who  are  inside  of  DoE,  or  outside?  Who  are  these  people? 

•  What  are  the  best  things  about  your  job? 

•  What  are  the  hardest  things  about  your  job? 

•  How  do  you  know  what  you  are  supposed  to  do  in  your  job?  Who  determines  your 
priorities? 

2.  Now,  I'd  like  to  talk  about  the  work  of  DoE  overall. 

•  From  your  perspective,  what  are  the  major  goals  and  priorities  of  DoE?  Who  sets  them? 

•  Do  any  goals  or  priorities  conflict? 

•  How  has  the  nature  of  work  at  DoE  changed? 
Prompt:  effect  of  Education  Reform 

•  What  makes  DoE's  work  difficult? 

•  What  could  make  DoE's  work  easier? 

3.  Now,  I  have  some  more  specific  questions  about  DoE  and  Education  Reform. 

The  page  I'm  giving  you  summarizes  1 1  key  components  of  the  Education  Reform  Act  of  1993. 
On  which  of  these  components  do  you  work  most?  (several  OK): 

Curriculum  standards 

Student  assessment  system 

Accountability  system  for  school  and  district  performance 

Local  govemance  and  management  of  schools  and  districts 
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Processes  for  certifying,  evaluating,  and  dismissing  school  personnel 

Professional  development 

School  finance/funding 

Choice  and  charter  schools 

Readiness  to  leam/early  childhood/literacy 

State-level  policy  and  planning 

Collecting  and  using  information  to  improve  system  performance 


In  the  areas  you  mention,  on  a  scale  of  1  to  10,  with  1  being  not  effective  and  10  being 
maximally  effective,  how  would  you  rate  the  effectiveness  of  implementation? 
•     To  what  would  you  attribute  that  degree  of  implementation  effectiveness?  (Prompt  for 
specifics  in  following  areas:) 

-  Data  and  information 

-  Technology 

-  Staff  (Prompt:  to  what  extend  does  your  division/cluster  rely  on  consultants? 
Do  you  see  the  use  of  consultants  as  a  problem?) 

-  Training 

-  Funds 

-  Communication  -  internal  and  external 
Other  resource  needs 


4.  Finally,  Fd  like  to  hear  your  ideas  about  what  changes  might  be  made  to  help  DoE  in  its 
work  implementing  Education  Reform. 

•  How  could  DoE  use  its  existing  resources  more  efficiently  to  meet  its  responsibilities? 

•  What  other  resources  firom  outside  of  DoE  (either  state  or  other)  could  help  DoE's  capacity? 

•  Are  there  other  states  or  models  of  managing  education  reform  that  you  find  interesting? 

•  Is  there  anything  else  you  think  I  should  know  about  DoE  or  the  capacity  of  the  state  to 
implement  Education  Reform? 

3.  Other  Stakeholders 

Same  introductory  material  as  in  other  protocols. 

Background  question  -How  does  your  job  connect  you  to  education  reform? 

1.  Massachusetts  Education  Reform  has  been  characterized  as  a  bargain  between 
the  state  and  local  districts,  in  which  the  state  provides  more  money  to 

districts  in  exchange  for  local  accountability  for  meeting  standards.  How  would 
you  characterize  that  description? 

2.  The  card  I'm  giving  you  summarizes  11  key  components  of  the  Education  Reform  Act  of 
1993.  Given  this  description  of  the  Act,  what  would  you  say  about  the  state's  capacity,  or 
ability,  to  implement  it?  What  has  the  state  done  well,  and  what  has  it  done  less  well?  (Ask 

APPENDIX  4  82 


g^^ 


respondents  to  specify  who  they  mean  when  they  refer  to  "the  state"-try  to  get  more 
specific  than  "DOE."  Also  prompt  if  necessary  about  different  types  of  responsibilities — 
technical  assistance,  regulation,  finance,  data  collection  and  information.) 

Curriculum  standards 

Student  assessment  system 

Accountability  system  for  school  and  district  performance 

Local  govemance  and  management  of  schools  and  districts 

Processes  for  certifying,  evaluating,  and  dismissing  school  personnel 

Professional  development 

School  finance/funding 

Choice  and  charter  schools 

Readiness  to  learn/early  childhood/literacy 

State-level  policy  and  planning 

Collecting  and  using  information  to  improve  system  performance 

4.  Why  has  the  state  done well? 

5.  Why  has  the  state  done ] less  well? 

6.  I'd  also  like  to  hear  your  perceptions  of  DoE's  goals  and  priorities.  In  particular,  how 
have  DoE's  goals  and  priorities  changed  over  time? 

Finally,  I'm  interested  in  hearing  your  recommendations  about  the  future  implementation 
of  Education  Reform. 

7.  What  advice  would  you  give  to  MERRC  about  the  state's  capacity  to  implement 
education  reform? 

8.  How  could  DoE  use  its  existing  resources  more  efficiently  to  meet  its  responsibilities? 

9.  How  could  other  agencies  or  outside  organizations  assist  DoE  with  implementing 
MERA? 

10.  How  could  your  organization  help  DoE  with  the  implementation  of  MERA?  Have  you 
been  approached  to  give  this  assistance?  Has  it  been  offered? 

11.  Are  there  other  states  or  models  of  managing  education  reform  that  you  find 
interesting? 

12.  Is  there  anything  else  you  think  I  should  know  about  the  capacity  of  the  state  to 
implement  Education  Reform? 


APPENDIX  4  83 


saoi 


Massachusetts'  State  Capacity  for  Educational  Reform: 
Superintendent/Central  Office  Survey 


ID//////////////  (superintendent  version) 


As  explained  in  the  cover  letter,  data  from  this  survey  will  be  used  to  assess  the  capacity  of  the 
Commonwealth  of  Massachusetts  to  carry  out  its  responsibilities  under  the  Education  Reform  Act  of 
1993.  The  research  is  sponsored  by  the  Massachusetts  Education  Reform  Review  Commission  and  is 
being  carried  out  by  the  Center  for  Education  Policy  of  the  School  of  Education  at  the  University  of 
Massachusetts,  Amherst. 

Your  perspective  as  a  district  administrator  is  particularly  important  to  us  in  identifying  areas  of  success  and  areas 
in  which  improvement  is  needed.  Your  responses  to  the  survey  will  be  confidential. 

The  report  will  be  completed  and  released  to  the  pubhc  by  the  Education  Reform  Review  Commission  later  this 
year.  Thank  you  for  participating! 


A.         Your  position 

1.  Please  indicate  which  description  most  closely  fits  your  position  (check  one); 
LJ  Superintendent 

U  Assistant  Superintendent 

Q  Other  (Specify) 

2.  What  percentage  of  your  student  population  lives  in  poverty  (as  determined  by  free  lunch?)_ 


B.         Your  own  experience  with  Education  Reform 

Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=strongly  agree;  2=agree;  3=disagree;  4=strongly  disagree;  DK=don't  know) 


Strongly 
Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

3.  I  understand  the  goals  of  the  Education  Reform  Act 

4.  The  state's  priorities  under  Education  Reform  are  clear. 

5.  State  policy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform  Act 
to  local  school  districts 


State  pohcy  makers  have  done  a  good  job  of 
commimicating  the  goals  of  the  Education  Reform  Act 
to  the  general  public 

The  state  has  reasonable  expectations  regarding  the  role  of 
districts  in  implementing  the  Education  Reform  Act 


,2 

,2 


.2 
,2 


.3. 
.3. 


.3. 
.3. 


.4. 

.4. 


.DK 
.DK 

.DK 

.DK 
.DK 
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8.  The  state  has  reasonable  expectations  regarding  the  role 

of  schools  in  implementing  the  Education  Reform  Act 1 2 3 4 DK 

9.  Education  Reform  has  had  a  positive  impact  on  my  district. .  1 2 3 4 DK 

Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=strongly  agree;  2=agree;  3=disagree;  4=strongly  disagree;  DK=don't  know) 


Strongly 
Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

10.  Education  Reform  has  improved  the  quality  of 
instruction  in  my  district 


1 1.  Education  Reform  has  improved  the  quality  of 
curriculum  in  my  district 


12.  Education  Reform  has  improved  the  quality  of 
professional  development  in  my  district 

13.  Education  Reform  fits  well  with  my 
district's  other  priorities 


14.  Education  Reform  has  crowded  out  other  initiatives 
I  think  are  more  important  

Which  ones? 


15.  The  Department  of  Education  collects  appropriate  data 
fi-om  schools 


16.  The  Department  of  Education  uses  these  data  appropriately . 

17.  The  data  my  district  gets  fi^om  the  Department  of 
Education  are  helpful  for  our  efforts  to  improve  the 
quality  of  education 


18.  When  I  have  a  question  about  Education  Reform,  the 
Department  of  Education  is  helpful  in  answering  it.  .., 

19.  The  Department  of  Education  seems  to  have  enough 
staff  to  carry  out  its  responsibilities 


20.  Department  of  Education  staff  members  have  the 
qualifications  and  experiences  they  need  to  provide  usefiil 
support  to  schools  and  districts 

21.  Under  Education  Reform,  the  state  has  too  much  power.  .. 


,2, 
.2 


.2, 
.2, 


.3. 
.3. 


.3. 
.3. 


.4. 
.4. 


.4. 
.4. 


.DK 


.DK 


.DK 


.DK 


.DK 


.DK 
.DK 

.DK 
.DK 
.DK 

.DK 
.DK 


22.  In  your  opinion,  are  there  components  of  the  Massachusetts  Education  Reform  Act  that  have  not  yet  been 
implemented?  If  so,  which  ones? 


23.  In  your  opinion,  does  the  state  make  contradictory  demands  on  schools?  Please  explain. 
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24.  Additional  Comments  About  Your  Own  Experience  with  Education  Reform? 


C.         Support  for  districts  and  schools 

How  would  you  rate  the  state's  support  for  your  school  in  the  following  areas? 

(Circle  the  appropriate  number:  l=excellent;  2=good;  3=fair;  4=poor;  NB=no  basis  for  judgement) 


Excellent 

Good 

Fair 

Poor 

No 
Basis 

25.  Aligning  curriculum  with  the  state's  frameworks 

26.  Providing  support  for  students  who  score  poorly  on  MCAS.. 

27.  Interpreting  MCAS  results 

28.  Developing  human  resource  policies  to  improve  staff  quality 

29.  Professional  development  for  teachers 

30.  Professional  development  for  administrators 

31.  Improving  schools'  overall  performance 

32.  Meeting  the  needs  of  special  populations 


,2, 
.2, 


.3. 
.3. 


.4. 
.4. 


.NB 
.NB 
.NB 
.NB 
.NB 
.NB 
.NB 
.NB 


33.  Do  you  beUeve  that  the  Department  of  Education  has  appropriate  human,  technical,  or  financial  resources  in 
order  to  carry  out  its  responsibilities  under  the  Education  Reform  Act?  Please  explain. 


34.  What  changes  would  you  recommend  regarding  how  the  Department  of  Education  works? 


35.  What  are  the  most  important  ways  for  the  Department  of  Education  to  be  involved  with  local  districts  and 
schools? 
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36.  Additional  Comments  about  Support  for  Districts  and  Schools 


D.         State  versus  local  responsibilities  in  Education  Reform 

The  Education  Reform  Act  of  1993  assigned  many  new  education-related  tasks  to  the  state  government.  For  each 
category  of  tasks  in  the  following  table,  please  indicate  whether  you  believe  these  tasks  should  be  primarily  a  state 
responsibility,  primarily  a  local  responsibility,  or  a  shared  responsibility. 


Task 

Primarily  State 
Responsibihty 

Shared 
Responsibility 

Primarily 

Local 

Responsibility 

Don't  Know 

37.  Developing  and  implementing 
curriculum  standards 

38.  Developing  and  administering  a 
student  assessment  system 

39.  Developing  an  accountability 
system  for  school  and  district 
performance 

40.  Supporting  local  govemance  and 
management  in  education  reform 

41.  Developing  and  administering 
processes  for  certifying, 
evaluating,  and  dismissing  school 
personnel 

42.  Providing  and/or  guiding 
professional  development 

43.  Allocating  funding  in  a  way  that 
is  adequate  and  equitable 

44.  Facilitating  inter-district  choice 

45.  Supervising  Charter  Schools 

46.  Ensuring  readiness  to  learn, 
through  early  childhood  and 
literacy  programs 

47.  Making  state-level  policy  and 
planning  decisions,  in 
coordination  with  key  actors 

48.  Collecting  and  using  information 
to  improve  the  performance  of  the 
state  educational  system 
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49.  Has  your  school  been  involved  in  Education  Reform-related  activities  with  state  agencies  or  offices  other  than 
the  Department  of  Education?  Which  ones? 


50.  Do  other  states  provide  models  of  managing  education  reform  that  you  believe  could  be  instructive?  Which 
states,  and  which  components  of  their  education  reform  models,  are  most  usefiil?  ? 


E.         Flow  of  Information 

51.  How  do  you  get  information  about  Education  Reform? 
(check  all  that  apply) 

LJ  Documents  mailed  directly  to  me  by  the  Department  of  Education 

LJ  Documents  mailed  directly  to  me  by  other  state  agencies  or  offices 

LJ  Documents  mailed  by  the  Department  to  my  district  and  circulated  to  the  schools 

LJ  The  Department  of  Education  web  site 

LJ  Via  a  union  or  professional  association 

LJ  The  media 

G  Other? 

52.  How  easy  is  it  for  you  to  get  information  from  the  Department  of  Education? 
LJ  Very  Easy 

LJ  Easy 

G  Difficult 

Q  Very  Difficult 
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53.  How  do  you  most  frequently  contact  someone  at  the  Department  of  Education? 

U  I  directly  contact  someone  one  I  already  know 

U  I  directly  contact  a  specific  department  or  program  area 

LJ  I  use  the  switchboard  operator  to  direct  me  to  the  appropriate  contact  person 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  call  them 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  send  them  an  email  message 

LJ  Other  (please  specifiy) 

LJ  I  do  not  contact  the  Department  of  Education 


F.  Assessing  Change 

54.  In  the  past  5  years,  what  changes  have  you  seen  in  your  school  with  respect  to  the  following: 


a.    Instructional  time 
Increase 


Decrease 


Remained  the  Same 


_#  minutes  per  week 


b.    Length  of  school  year  or  day 
Increase         Decrease 


Remained  the  Same 


.  #  hours/days 


c.    Teacher  attendance 

Increase         Decrease 


Remained  the  Same 


%change,  if  any 


d.    Teachers  with  appropriate  certification 
^Increase         Decrease 


Remained  the  Same 


_%change,  if  any 


e.    Percent  of  budget  allocated  to  professional  development 

^Increase         ^Decrease  Remained  the  Same 


%change,  if  any 


f.     Percent  of  budget  allocated  to  curriculum  and  instruction  (texts,  new  programs,  etc.) 

Increase         ^Decrease  ^Remained  the  Same  %change,  if  any 

If  you  have  additional  comments,  please  write  them  in  the  space  below. 
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Massachusetts^  State  Capacity  for  Educational  Reform:  Principal  Survey 


TDIIIIIIIIIIIIII  (principal  version) 


As  explained  in  the  cover  letter,  data  from  this  survey  will  be  used  to  assess  the  capacity  of  the  Commonwealth  of 
Massachusetts  to  carry  out  its  responsibilities  under  the  Education  Reform  Act  of  1993.  The  research  is  sponsored 
by  the  Massachusetts  Education  Reform  Review  Commission  and  is  being  carried  out  by  the  Center  for  Education 
Policy  of  the  School  of  Education  at  the  University  of  Massachusetts,  Amherst. 

Your  perspective  as  a  principal  is  particularly  important  to  us  in  identifying  areas  of  success  and  areas  in  which 
improvement  is  needed.  Your  responses  to  tiie  survey  will  be  confidential. 

The  report  will  be  completed  and  released  to  the  public  by  the  Education  Reform  Review  Commission  later  this 
year.  Thank  you  for  participating! 


A.  Your  position 

1.  Please  indicate  which  description  most  closely  fits  your  position  (check  one).- 
LJ  Principal 

LJ  Assistant  or  Vice  Principal 

Q  Other  (Specify) 

2.  What  percentage  of  your  student  population  lives  in  poverty  (as  determined  by  free  lunch?) 

B.  Your  own  experience  with  Education  Reform 

Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=sfrongly  agree;  2=agree;  3=disagree;  4=sti-ongly  disagree;  DK=don't  know) 


Strongly 
Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

3.  I  imderstand  the  goals  of  the  Education  Reform  Act 1 2 3 4 DK 

4.  The  state's  priorities  under  Education  Reform  are  clear 1 2 3 4 DK 

5.  State  policy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform  Act 
to  local  school  districts 


1 


8.    The  state  has  reasonable  expectations  regarding  the  role 
of  schools  in  implementing  the  Education  Reform  Act. 


.DK 


6.  State  poUcy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform  Act 

to  the  general  public 1 2 3 4 DK 

7.  The  state  has  reasonable  expectations  regarding  the  role  of 

districts  in  implementing  the  Education  Reform  Act 1 2 3 4 DK 


.DK 
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9.    Education  Reform  has  had  a  positive  impact  on  my  district. .  1 2 3 4. 


.DK 


Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=strongly  agree;  2=agree;  3=disagree;  4=strongly  disagree;  DK=don't  know) 


Strongly 
Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

10.  Education  Reform  has  had  a  positive  impact  on  my  school. 

11.  Education  Reform  has  improved  the  quality  of 
instruction  in  my  district 


12.  Education  Reform  has  improved  the  quality  of 
curriculum  in  my  district 


13.  Education  Reform  has  improved  the  quality  of 
professional  development  in  my  district 


14.  Education  Reform  fits  well  with  my 

district's  other  priorities 

15.  Education  Reform  has  crowded  out  other  initiatives 
I  think  are  more  important  

Which  ones? 


16.  The  Department  of  Education  collects  appropriate  data 
from  schools 


17.  The  Department  of  Education  uses  these  data  appropriately. 

18.  The  data  my  district  gets  from  the  Department  of 
Education  are  helpfiil  for  our  efforts  to  improve  the 
quality  of  education 


19.  When  I  have  a  question  about  Education  Refomi,  the 
Department  of  Education  is  helpfril  in  answering  it.  .. 

20.  The  Department  of  Education  seems  to  have  enough 
staff  to  carry  out  its  responsibilities 


21.  Department  of  Education  staff  members  have  the 

qualifications  and  experiences  they  need  to  provide  usefiil 
support  to  schools  and  districts 


22.  Under  Education  Reform,  the  state  has  too  much  power. 


,2, 
,2, 


.2, 
,2, 

,2, 
,2. 


.3. 
.3. 


.3. 

.3. 


.3. 
.3. 


.4. 
.4. 


.4. 
.4. 


.DK 
.DK 
.DK 
.DK 
.DK 
.DK 


.DK 
.DK 

.DK 
.DK 
.DK 

.DK 
.DK 


23.  In  your  opinion,  are  there  components  of  the  Massachusetts  Education  Reform  Act  that  have  not  yet  been 
implemented?  If  so,  which  ones? 


24.  In  your  opinion,  does  the  state  make  contradictory  demands  on  schools?  Please  explain. 
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25.  Additional  Comments  About  Your  Own  Experience  with  Education  Reforni? 


C.         Support  for  districts  and  schools 

How  would  you  rate  the  state's  support  for  your  school  in  the  following  areas? 

(Circle  the  appropriate  number:  l=excellent;  2=good;  3=fair;  4=poor;  NB=no  basis  for  judgement) 


Excellent 

Good 

Fair 

Poor 

No 
Basis 

26.  Aligning  curriculum  with  the  state's  frameworks 

27.  Providing  support  for  students  who  score  poorly  on  MCAS.. 

28.  Interpreting  MCAS  results 

29.  Developing  human  resource  policies  to  improve  staff  quality 

30.  Professional  development  for  teachers 

3 1 .  Professional  development  for  administrators 

32.  Improving  schools'  overall  performance 

33.  Meeting  the  needs  of  special  populations 


.2 
,2 


.3. 
.3. 


.4. 
.4. 


.NB 
.NB 

.NB 
.NB 
.NB 
.NB 
.NB 
.NB 


34.  Do  you  beUeve  that  the  Department  of  Education  has  appropriate  human,  technical,  or  financial  resources  in 
order  to  cany  out  its  responsibilities  under  the  Education  Reform  Act?  Please  explain. 


35.  What  changes  would  you  recommend  regarding  how  the  Department  of  Education  works? 


36.  What  are  the  most  important  ways  for  the  Department  of  Education  to  be  involved  with  local  districts  and 
schools? 
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37.  Additional  Comments  about  Support  for  Districts  and  Schools 


D.         State  versus  local  responsibilities  in  Education  Reform 

The  Education  Reform  Act  of  1993  assigned  many  new  education-related  tasks  to  the  state  government.  For  each 
category  of  tasks  in  the  following  table,  please  indicate  whether  you  beheve  these  tasks  should  be  primarily  a  state 
responsibility,  primarily  a  local  responsibility,  or  a  shared  responsibiUty. 


Task 

Primarily  State 
Responsibility 

Shared 
Responsibility 

Primarily 

Local 

Responsibility 

Don't  Know 

38.  Developing  and  implementing 
curriculum  standards 

39.  Developing  and  administering  a 
student  assessment  system 

40.  Developing  an  accountability 
system  for  school  and  district 
perfoniiance 

41.  Supporting  local  governance  and 
management  in  education  reform 

42.  Developing  and  administering 
processes  for  certifying, 
evaluating,  and  dismissing  school 
personnel 

43.  Providing  and/or  guiding 
professional  development 

44.  Allocating  fimding  in  a  way  that 
is  adequate  and  equitable 

45.  Facilitating  inter-district  choice 

46.  Supervising  Charter  Schools 

47.  Ensuring  readiness  to  leam, 
through  early  childhood  and 
Uteracy  programs 

48.  Making  state-level  poUcy  and 
planning  decisions,  in 
coordination  with  key  actors 

49.  Collecting  and  using  information 
to  improve  the  performance  of  the 
state  educational  system 
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50.  Has  your  school  been  involved  in  Education  Reform-related  activities  with  state  agencies  or  offices  other  than 
the  Department  of  Education?  Which  ones? 


51.  Do  other  states  provide  models  of  managing  education  reform  that  you  beheve  could  be  instructive?  Which 
states,  and  which  components  of  their  education  reform  models,  are  most  useful?  ? 


£.  Flow  of  Information 

52.  How  do  you  get  information  about  Education  Reform? 
(check  all  that  apply) 

U  Documents  mailed  directly  to  me  by  the  Department  of  Education 

LJ  Documents  mailed  directly  to  me  by  other  state  agencies  or  offices 

LJ  Documents  mailed  by  the  Department  to  my  district  and  circulated  to  the  schools 

LJ  The  Department  of  Education  web  site 

U  Via  a  union  or  professional  association 

LJ  The  media 

G  Other? 

53.  How  easy  is  it  for  you  to  get  information  firom  the  Department  of  Education? 
U  Very  Easy 

LJ  Easy 

Q  Difficult 

Q  Very  Difficult 


APPENDIX  4  94 


54.  How  do  you  most  frequently  contact  someone  at  the  Department  of  Education? 

LJ  I  directly  contact  someone  one  I  already  know 

LJ  I  directly  contact  a  specific  department  or  program  area 

LJ  I  use  the  switchboard  operator  to  direct  me  to  the  appropriate  contact  person 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  call  them 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  send  them  an  email  message 

LJ  Other  (please  specifiy) 

LJ  I  do  not  contact  the  Department  of  Education 


F.  Assessing  Change 

55.  In  the  past  5  years,  what  changes  have  you  seen  in  your  school  with  respect  to  the  following: 

g.    Instmctional  time 

Increase         ^Decrease  Remained  the  Same  #  minutes  per  week 


h.    Length  of  school  year  or  day 
Increase         Decrease 


Remained  the  Same 


,  #  hours/days 


L     Teacher  attendance 

Increase         Decrease 


Remained  the  Same 


%change,  if  any 


j.     Teachers  with  appropriate  certification 
^Increase         ^Decrease 


Remained  the  Same 


_%change,  if  any 


k.    Percent  of  budget  allocated  to  professional  development 

Increase         ^Decrease  Remained  the  Same 


%change,  if  any 


L     Percent  of  budget  allocated  to  curriculum  and  instruction  (texts,  new  programs,  etc.) 

^Increase         ^Decrease  Remained  the  Same  %change,  if  any 

If  you  have  additional  comments,  please  write  them  in  the  space  below. 
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Massachusetts'  State  Capacity  for  Educational  Reform:  Teacher  Survey 


ID#######  (teacher  version) 


As  explained  in  the  cover  letter,  data  from  this  survey  will  be  used  to  assess  the  capacity  of  the  Commonwealth  of 
Massachusetts  to  carry  out  its  responsibilities  under  the  Education  Reform  Act  of  1993.  The  research  is  sponsored 
by  the  Massachusetts  Education  Reform  Review  Commission  and  is  being  carried  out  by  the  Center  for  Education 
Policy  of  the  School  of  Education  at  the  University  of  Massachusetts,  Amherst. 

Your  perspective  as  a  teacher  is  particularly  important  to  us  in  identifying  areas  of  success  and  areas  in  which 
improvement  is  needed.  Your  responses  to  the  survey  will  be  confidential. 

The  report  will  be  completed  and  released  to  the  public  by  the  Education  Reform  Review  Commission  later  this 
year.  Thank  you  for  participating! 


A.         Your  position 

1.    Please  indicate  which  description  most  closely  fits  your  position: 

LJ  Elementary  school  teacher 
Grade  leveP 

LJ  Middle  school  teacher 

Subject  area(s)? 

LJ  High  school  teacher 

Subject  area(s)? 

\3  Other  (Specify) 


B.         Your  own  experience  with  Education  Reform 

Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=strongly  agree;  2=agree;  3=disagree;  4=strongly  disagree;  DK=don't  know) 


Strongly 

Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

2.  I  understand  the  goals  of  the  Education  Reform  Act 

3.  The  state's  priorities  under  Education  Reform  are  clear.  ..., 

4.  State  policy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform  Act 

to  local  school  districts 

5.  State  poUcy  makers  have  done  a  good  job  of 
communicating  the  goals  of  the  Education  Reform  Act 

to  the  general  public 

6.  The  state  has  reasonable  expectations  regarding  the  role  of 
districts  in  implementing  the  Education  Refomi  Act 

7.  The  state  has  reasonable  expectations  regarding  the  role 
of  schools  in  implementing  the  Education  Reform  Act 
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8.    Education  Reform  has  had  a  positive  impact  on  my  district.  .  1 2 3 4. 


.DK 


Please  indicate  the  extent  to  which  you  disagree  or  agree  with  the  following  statements: 

(Circle  the  appropriate  number:  l=strongly  agree;  2=agree;  3=disagree;  4=strongly  disagree;  DK=don't  know) 


Strongly 
Agree 

Agree 

Disagree 

Strongly 
Disagree 

Don't 
Know 

9.  Education  Reform  has  had  a  positive  impact  on  my  school. 

10.  Education  Reform  has  improved  the  quality  of 
instruction  in  my  district 


1 1.  Education  Reform  has  improved  the  quality  of 
curriculum  in  my  district 


12.  Education  Reform  has  improved  the  quality  of 
professional  development  in  my  district 

13.  Education  Reform  fits  well  with  my 

district's  other  priorities 

14.  Education  Reform  has  crowded  out  other  initiatives 
I  think  are  more  important  

Which  ones? 


15.  The  data  my  district  gets  from  the  Department  of 
Education  are  helpful  for  our  efforts  to  improve  the 
quality  of  education 

16.  When  I  have  a  question  about  Education  Reform,  the 
Department  of  Education  is  helpful  in  answering  it.  ... 


17.  The  Department  of  Education  seems  to  have  enough 
staff  to  carry  out  its  responsibilities 

18.  Department  of  Education  staff  members  have  the 
qualifications  and  experiences  they  need  to  provide  useful 
support  to  schools  and  districts 

19.  Under  Education  Reform,  the  state  has  too  much  power.  .. 


,2, 
.2, 


,2, 

,2. 


.3. 
.3. 


.3. 
.3. 


.4. 
.4. 


.4. 

.4. 


.DK 


.DK 


.DK 


.DK 


.DK 


.DK 


.DK 
.DK 

.DK 

.DK 
.DK 


20.  In  your  opinion,  are  there  components  of  the  Massachusetts  Education  Reform  Act  that  have  not  yet  been 
implemented?  ff  so,  which  ones? 


21.  In  your  opinion,  does  the  state  make  contradictory  demands  on  schools?  Please  explain. 


22.  Additional  Comments  About  Your  Own  Experience  with  Education  Reform? 
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C.         Support  for  districts  and  schools 


How  would  you  rate  the  state's  support  for  your  school  in  the  following  areas? 

(Circle  the  appropriate  number:  l=excellent;  2=good;  3=fair;  4=poor;  NB=no  basis  for  judgement) 


Excellent 

Good 

Fair 

Poor 

No 
Basis 

23.  Aligning  curriculum  with  the  state's  frameworks 

24.  Providing  support  for  students  who  score  poorly  on  MCAS.. 

25.  Interpreting  MCAS  results 

26.  Developing  human  resource  policies  to  improve  staff  quality 

27.  Professional  development  for  teachers 

28.  Professional  development  for  administrators 

29.  Improving  schools'  overall  performance 

30.  Meeting  the  needs  of  special  populations 


,2, 
.2, 


.3. 
.3. 


.4. 
.4. 


.NB 
.NB 
.NB 
.NB 
.NB 
.NB 
.NB 
.NB 


31.  Do  you  believe  that  the  Department  of  Education  has  appropriate  human,  technical,  or  financial  resources  in 
order  to  carry  out  its  responsibilities  under  the  Education  Reform  Act?  Please  explain. 


32.  What  changes  would  you  recommend  regarding  how  the  Department  of  Education  works? 


33.  What  are  the  most  important  ways  for  the  Department  of  Education  to  be  involved  with  local  districts  and 
schools? 


34.  Additional  Comments  about  Support  for  Districts  and  Schools 
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D.         State  versus  local  responsibilities  in  Education  Reform 

The  Education  Reform  Act  of  1993  assigned  many  new  education-related  tasks  to  the  state  government.  For  each 
category  of  tasks  in  the  following  table,  please  indicate  whether  you  believe  these  tasks  should  be  primarily  a  state 
responsibility,  primarily  a  local  responsibility,  or  a  shared  responsibility. 


Task 

Primarily  State 
Responsibility 

Shared 
Responsibility 

Primarily 

Local 

Responsibility 

Don't  Know 

35.  Developing  and  implementing 
curriculum  standards 

36.  Developing  and  administering  a 
student  assessment  system 

37.  Developing  an  accoimtability 
system  for  school  and  district 
performance 

38.  Supporting  local  govemance  and 
management  in  education  reform 

39.  Developing  and  administering 
processes  for  certifying, 
evaluating,  and  dismissing  school 
personnel 

40.  Providing  and/or  guiding 
professional  development 

41.  Allocating  funding  in  a  way  that 
is  adequate  and  equitable 

42.  Facilitating  inter-district  choice 

43.  Supervising  Charter  Schools 

44.  Ensuring  readiness  to  learn, 
through  early  childhood  and 
literacy  programs 

45.  Making  state-level  pohcy  and 
planning  decisions,  in 
coordination  with  key  actors 

46.  Collecting  and  using  information 
to  improve  the  performance  of  the 
state  educational  system 

47.  Has  your  school  been  involved  in  Education  Reform-related  activities  with  state  agencies  or  offices  other  than 
the  Department  of  Education?  Which  ones? 
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48.  Do  other  states  provide  models  of  managing  education  reform  that  you  beheve  could  be  instructive?  Which 
states,  and  which  components  of  their  education  reform  models,  are  most  useful?  ? 


£.         Flow  of  Information 

49.  How  do  you  get  information  about  Education  Reform? 
(check  all  that  apply) 

U  Documents  mailed  directly  to  me  by  the  Department  of  Education 

LJ  Documents  mailed  directly  to  me  by  other  state  agencies  or  offices 

LJ  Documents  mailed  by  the  Department  to  my  district  and  circulated  to  the  schools 

LJ  The  Department  of  Education  web  site 

U  Via  a  union  or  professional  association 

LJ  The  media 

G  Other? 

50.  How  easy  is  it  for  you  to  get  information  from  the  Department  of  Education? 
LJ  Very  Easy 

LJ  Easy 

Q  Difficult 

Q  Very  Difficult 
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51.  How  do  you  most  frequently  contact  someone  at  the  Department  of  Education? 

LJ  I  directly  contact  someone  one  I  already  know 

LJ  I  directly  contact  a  specific  department  or  program  area 

LJ  I  use  the  switchboard  operator  to  direct  me  to  the  appropriate  contact  person 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  call  them 

LJ  I  use  the  website  to  find  the  appropriate  contact  person  and  I  send  them  an  email  message 

LJ  Other  (please  specifiy) 

LJ  I  do  not  contact  the  Department  of  Education 

If  you  have  additional  comments,  please  write  them  in  the  space  below. 
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Appendix  5 

List  of  Interviewees 


Virginia  Anderson,  Associate  Executive  Secretary,  Massachusetts  Secondary  School  Administrators ' 

Association 

Robert  Antonucci,  President,  Harcourt  Learning  Direct;  Former  Commissioner  of  Education 

Charles  Baker,  President,  Harvard  Pilgrim  Health  Care;  Former  Secretary  of  Administration  and  Finance; 

Member,  Massachusetts  Board  of  Education 

Susan  Miller  Barker,  Associate  Commissioner  for  Charter  Schools,  Massachusetts  Department  of  Education 

Noah  Berger,  Council  to  the  President,  Office  of  Massachusetts  Senate  President  Thomas  Birmingham 

Mary  Brabeck,  Dean,  Boston  College  School  of  Education;  Member  of  Massachusetts  Education  Reform 

Review  Commission 

Cecelia  Buckley,  Director  of  Professional  Development,  Hampshire  Educational  Collaborative 

Andrew  Calkins,  Executive  Director,  Masslnsight  Education 

Robert  Costrell,  Director  of  Research  and  Development,  Massachusetts  Office  of  Administration  and 

Finance 

Anthony  DeLorenzo,  Director  of  Budget  and  Finance,  Massachusetts  Department  of  Education 

Ruth  Derfler,  Director  of  Human  Resources,  Massachusetts  Department  of  Education 

Juliane  Dovj,  Associate  Commissioner  for  Accountability  and  Targeted  Assistance,  Massachusetts 

Department  of  Education 

David  Driscoll,  Commissioner  of  Education 

Ann  Duffy,  Associate  Commissioner  for  Educator  Quality,  Massachusetts  Department  of  Education 

William  Edgerly,  President,  Business  for  Better  Schools 

Joseph  ^gzn.  Assistant  Budget  Director,  Massachusetts  Office  of  Administration  and  Finance 

Tyler  Fairbank,  President,  MassExcellence 

Peter  Finn,  Senior  Vice  President  for  Education,  Harcourt  Learning  Direct;  Former  Director, 

Massachusetts  Association  of  School  Superintendents 

Sue  Freedman,  President,  Teachers  21;  Previously  with  Massachusetts  Department  of  Education 

Dan  French,  Executive  Director,  Center  for  Collaborative  Education;  Previously  with  Massachusetts 

Department  of  Education 

Barbara  Gardner,  Associate  Commissioner  for  School  Readiness,  Department  of  Education;  Former 

Majority  Whip,  Massachusetts  House  of  Representatives 

Joseph  Giannino,  Director  of  Legislative  Affairs,  Massachusetts  Department  of  Education 

Judith  Gill,  Chancellor,  Massachusetts  Board  of  Higher  Education 

Stephen  Gorrie,  President,  Massachusetts  Teachers '  Association 

Ellen  Guiney,  Executive  Director,  Boston  Plan  for  Excellence 

James  Hamos,  Senior  Associate,  Academic  Affairs,  Office  of  the  President,  University  of  Massachusetts 

James  Hardy,  Field  Service  Representative,  Massachusetts  Association  of  School  Committees 

Frank  Haydu,  Former  Interim  Commissioner  of  Education 

Nadya  Aswad  Higgins,  Executive  Director,  Massachusetts  Elementary  School  Principals '  Association 

Sylvester  Ingeme,  Director  of  Corporate  Contributions,  FleetBoston  Foundation 

Bailey  Jackson,  Dean,  School  of  Education,  University  of  Massachusetts  Amherst 

Tripp  Jones,  Executive  Director,  MassINC 

Francis  Kane,  Associate  Commissioner,  School  to  Career,  Massachusetts  Department  of  Education 

Martin  Kaplan,  Senior  Partner,  Hale  and  Dorr  LLP;  Former  Chair,  Massachusetts  Board  of  Education 

Kathleen  Y^QWey,  President,  Massachusetts  Federation  of  Teachers 

Kathe  YJaVxudn,  Evaluation  and  Research  Specialist,  Student  Assessment  Systems,  Massachusetts 

Department  of  Education 

Lyle  Kirtman,  President,  Future  Management  Systems 

Glenn  Koochcr,  Executive  Director,  Massachusetts  Association  of  School  Committees 

Harold  Lane,  Executive  Director,  Alliance  for  Education;  Former  Co  Chair,  Committee  on  Education,  Arts, 

and  Humanities,  Massachusetts  General  Court 
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Kenneth  Lemanski,  Special  Assistant  to  the  Provost,  University  of  Massachusetts  Lowell,  Former  member 

of  Massachusetts  House  of  Representatives 

David  Lizotte,  Lecturer,  Director,  Teacher  Certification  Program,  College  of  the  Holy  Cross 

Christopher  Martes,  Executive  Director,  Massachusetts  Association  of  School  Superintendents 

Linda  Martin,  Director  of  Reading,  Massachusetts  Department  of  Education 

Meg  Mayo-Brown,  Director  of  New  Educators,  Massachusetts  Department  of  Education 

Kathryn  McHugh,  Jessie  B.  Cox  Charitable  Trust 

Beverly  Mxyaxes,  Researcher,  Massachusetts  Teachers'  Association 

Jeffrey  Nellhaus,  Associate  Commissioner  for  Student  Assessment  Programs,  Massachusetts  Department  of 

Education 

Thomas  Noonan,  Associate  Commissioner  of  Mathematics,  Science,  and  Technology/Engineering, 

Massachusetts  Department  of  Education 

Pendred  Noyce,  Trustee,  Noyce  Foundation,  Member  of  Massachusetts  Education  Reform  Review 

Commission 

James  Peyser,  Chair,  Massachusetts  Board  of  Education;  Chair,  Education  Management  and  Accountability 

Council;  Education  Advisor  to  Governor  Jane  Swift 

Noel  Pixley,  Principal,  Thornton  Burgess  Middle  School,  Hampden,  MA 

Darrell  Pressley,  Director  of  Community  Outreach,  Massachusetts  Department  of  Education 

Paul  Reville,  Chair,  Massachusetts  Education  Reform  Review  Commission;  Director,  Pew  Forum;  Former 

Member,  Massachusetts  Board  of  Education 

Constance  Rizoli,  Research  Director,  Joint  Committee  on  Education,  Arts,  andHumanties,  Massachusetts 

General  Court 

Mark  Roosevelt,  President,  Massachusetts  Biomedical  Initiatives;  Member,  Education  Management 

Accountability  Board;  Former  Co-Chair,  Joint  Committee  on  Education,  Arts,  and  Humanities, 

Massachusetts  General  Court 

Alan  Safran,  Deputy  Commissioner  for  Administration  and  Policy,  Massachusetts  Department  of  Education 

Roberta  Schaefer,  Executive  Director,  Worcester  Regional  Research  Bureau;  Member,  Massachusetts 

Board  of  Education 

Rhoda  Schneider,  Senior  Associate  Commissioner  and  General  Counsel,  Massachusetts  Department  of 

Education 

Joan  Schuman,  Executive  Director,  Hampshire  Educational  Collaborative,  Previously  with  Department  of 

Education 

Michael  Sentance,  Education  Advisor  to  Former  Governor  Paul  Cellucci 

Sylvia  Smith,  Chief  of  Staff,  Office  of  Massachusetts  Senator  Robert  A.  Antonioni 

Barbara  Solomon,  Title  I  Director,  Massachusetts  Department  of  Education 

John  Stager,  Director  of  Program  Quality  Assurance,  Massachusetts  Department  of  Education 

Sandra  Stotsky,  Deputy  Commissioner  for  Academic  Affairs,  Massachusetts  Department  of  Education 

Patricia  Sweitzer,  Consultant,  MassPartners 

Elaine  Taber,  Director  of  Policy  Development,  Office  of  Massachusetts  House  Speaker  Thomas  Finneran 

Carole  Thomson,  Associate  Commissioner  for  Education  Program  Services,  Massachusetts  Department  of 

Education 

Kathryn  Viator,  Manager  of  Student  Assessment,  Massachusetts  Department  of  Education 

Dieter  Wahl,  Director,  Bureau  of  Education  Audit 

Susan  Wheltle,  Coordinator,  Office  of  Humanities,  Massachusetts  Department  of  Education 

Melanie  Winklosky,  Special  Assistant  to  the  Massachusetts  Board  of  Education 

Margaret  Wood,  Policy  Analyst,  State  Board  of  Higher  Education;  Formerly  with  Joint  Commission  on 

Educator  Preparation 

Jeffrey  Wulfson,  Chief  Financial  Officer,  Massachusetts  Department  of  Education 

Francis  Zak,  Principal,  Ralph  Mahar  Regional  High  School,  Orange,  MA 
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Appendix  6 

Department  of  Education  Consultants,  FY  1995— FY  2000 

Analysis  for  these  tables  was  conducted  by  Susan  Bowles  and  Kathryn  McDermott.  The  data  came  from 
the  Massachusetts  Department  of  Education. 


Accountability 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 

— 

~ 

— 

— 

1996 

0 

~ 

— 

— 

.. 

1997 

0 

— 

~ 

— 

~ 

1998 

0 

~ 

~ 

— 

— 

1999 

4 

$  7,755.00 

$    1,878.50 

$     4,347.42 

$17,389.68 

2000 

2 

8,572.92 

31,290.00 

19,931.46 

39,862.92 

Six-Year  Total 

57,252.60 

Administrat 

on  and  Policy 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

~ 

~ 

~ 

$  15,206.50 

1996 

0 

~ 

~ 

~ 

~ 

1997 

0 

— 

— 

~ 

— 

1998 

0 

~ 

~ 

~ 

~ 

1999 

0 

— 

— 

~ 

~ 

2000 

0 

— 

— 

~ 

~ 

Six- Year  Total 

$  15,206.50 

Adult  and  Community  Learning 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

7 

$  42,911.80 

$  8,345.99 

$  18,482.35 

$    129,376.47 

1996 

5 

40,627.90 

26,429.00 

30,474.39 

152,371.94 

1997 

9 

47,336.00 

2,176.20 

30,288.34 

272,595.02 

1998 

8 

45,631.35 

2,176.20 

26,587.45 

212,699.56 

1999 

4 

149,747.50 

1,440.00 

62,388.66 

249,554.63 

2000 

4 

27,129.60 

16,500.00 

24,216.75 

96,867.00 

Six- Year  Total 

1,113,464.62 
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Certification  Administration 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

19 

$  35,965.57 

$  6,216.69 

$   16,726.07 

$     317,7.95.26 

1996 

19 

40,756.30 

7,714.00 

21,464.32 

407,822.09 

1997 

23 

45,140.00 

2,533.80 

25,668.66 

590,379.24 

1998 

22 

47,128.60 

502.95 

28,562.67 

628,378.77 

1999 

19 

50,631.30 

712.25 

19,722.75 

374,732.27 

2000 

18 

28,133.51 

780.00 

13,623.14 

245,216.51 

Six-Year  Total 

2,564,324.14 

Charter  Schools 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

~ 

— 

$   16,852.49 

1996 

0 

— 

~ 

~ 

~ 

1997 

1 

~ 

~ 

~ 

44,603.41 

1998 

0 

~ 

~ 

— 

~ 

1999 

0 

~ 

— 

— 

— 

2000 

0 

~ 

— 

— 

— 

Six-Year  Total 

61,455.90 

Chief  Financial  Officer 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 

— 

— 

~ 

— 

1996 

0 

~ 

~ 

~ 

— 

1997 

1 

— 

~ 

— 

$  37,162.80 

1998 

0 

— 

— 

~ 

— 

1999 

0 

— 

— 

~ 

— 

2000 

0 

— 

~ 

— 

~ 

Six- Year  Total 

37,162.80 

Chief  Technology  Officer 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Pajmient 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

~ 

~ 

— 

$      28,432.98 

1996 

1 

— 

~ 

— 

40,331.94 

1997 

0 

— 

~ 

— 

— 

1998 

2 

$  40,510.79 

$  17,119.44 

28,815.12 

57,630.23 

1999 

3 

41,587.69 

6,829.18 

20,058.74 

60,176.22 

2000 

3 

27,218.20 

17,677.11 

22,541.30 

67,623.91 

Six- Year  Total 

254,195.28 
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Commissioner's  Office 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

— 

~ 

S     34,127.97 

1996 

0 

— 

~ 

~ 

— 

1997 

3 

$  62,092.00 

$  6,100.00 

$     32,741.10 

98,223.29 

1998 

2 

13,300.00 

403.75 

6,851.88 

13,703.75 

1999 

7 

41,785.34 

3,820.00 

25,990.89 

181,936.23 

2000 

5 

27,466.80 

11,257.31 

21,597.58 

107,987.89 

Six- Year  Total 

435,979.13 

Early  Chiidh 

lood 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 

— 

~ 

— 

— 

1996 

0 

~ 

— 

— 

~ 

1997 

0 

~ 

_ 

.- 

~ 

1998 

0 

~ 

~ 

~ 

— 

1999 

3 

$    3,177.50 

$    300.00 

$  2,169.08 

$    6,507.23 

2000 

3 

22,671.25 

1,277.25 

7,557.08 

22,671.25 

Six-Year  Total 

29,178.48 

Education  Program  Services 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

4 

$20,482.12 

$  3,166.25 

$7,709.18 

$  30,836.73 

1996 

0 

— 

-- 

~ 

~ 

1997 

0 

— 

~ 

~ 

~ 

1998 

0 

— 

— 

— 

— 

1999 

1 

— 

— 

~ 

3,558.21 

2000 

0 

— 

~ 

- 

~ 

Six-Year  Total 

34,394.94 

Educator  Preparation  in  Higher  Education 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Pajonent 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 

— 

— 

~ 

~ 

1996 

4 

$  38,070.60 

$    2,249.90 

$  21,547.85 

$  86,191.38 

1997 

3 

41,641.50 

12,545.38 

24,752.71 

74,258.14 

1998 

2 

42,399.04 

39,997.44 

41,198.24 

82,396.48 

1999 

1 

~ 

— 

— 

44,869.44 

2000 

1 

— 

— 

~ 

26,814.16 

Six- Year  Total 

314,529.60 
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Humanities 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

2 

$  29,633.63 

$  28,579.69 

$  29,106.66 

S  58,213.32 

1996 

0 

— 

~ 

— 

— 

1997 

0 

— 

— 

~ 

~ 

1998 

0 

— 

— 

— 

— 

1999 

0 

— 

— 

~ 

— 

2000 

0 

~ 

— 

— 

— 

Six-Year  Total 

58,213.32 

Learning  Support  Servic< 

3S 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

13 

$  44,672.50 

$  8,229.00 

$  28,780.46 

$    374,145.96 

1996 

9 

44,472.00 

11,667.00 

31,637.12 

284,734.08 

1997 

2 

43,073.92 

30,167.80 

36,620.86 

73,241.72 

1998 

6 

29,585.15 

10,914.48 

19,500.23 

117,001.40 

1999 

8 

44,051.35 

2,725.51 

33,610.26 

268,882.05 

2000 

7 

26,059.20 

19,934.80 

23,703.26 

165,922.84 

Six- Year  Total 

1,283,928.05 

Legal  Office 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

— 

— 

$    9,350.00 

1996 

1 

— 

— 

— 

19,860.00 

1997 

2 

$23,174.06 

$  1,800.00 

$  12,487.03 

24,974.06 

1998 

1 

— 

~ 

— 

1,500.00 

1999 

1 

— 

~ 

~ 

2,025.00 

2000 

0 

~ 

~ 

~ 

~ 

Six-Year  Total 

57,709.06 

Legislative  Affairs 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 







— 

1996 

7 

$    36,915.00 

$    7,939.40 

$    22,548.66 

$  157,840.65 

1997 

6 

44,999.50 

21,475.70 

39,240.41 

235,442.43 

1998 

4 

46,347.80 

8,800.32 

36,067.72 

144,270.87 

1999 

2 

48,779.74 

4,788.20 

26,783.97 

53,567.94 

2000 

2 

28,521.55 

27,111.64 

27,816.60 

55,633.19 

Six-Year  Total 

646,755.08 
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Math  and  Science 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

4 

$  43,893.58 

$26,439.19 

$  32,840.15 

$     131,360.58 

1996 

7 

43,554.09 

8,870.80 

30,819.16 

215,734.14 

1997 

5 

46,225.80 

32,430.96 

38,815.13 

194,075.64 

1998 

7 

46,896.80 

20,660.80 

34,052.42 

238,366.95 

1999 

7 

48,906.83 

26,679.77 

42,528.06 

297,696.44 

2000 

7 

27,719.59 

4,023.24 

19,186.28 

134,303.94 

Six- Year  Total 

1,211,537.69 

Media  Reiat  ons 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
PajTnent 

Total 

Consultant 

Payments 

1995 

1 

~ 

— 

.- 

$     2,074.00 

1996 

2 

$  44,564.64 

$  30,397.37 

$  37,481.01 

74,962.01 

1997 

1 

~ 

— 

— 

23,607.60 

1998 

0 

~ 

~ 

— 

~ 

1999 

1 

— 

— 

— 

558.00 

2000 

0 

— 

— 

— 

~ 

Six- Year  Total 

101,201.61 

Nutrition 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

~ 

~ 

$  22,288.00 

1996 

1 

~ 

— 

~ 

18,624.00 

1997 

0 

— 

— 

~ 

— 

1998 

1 

— 

~ 

~ 

19,340.28 

1999 

1 

— 

— 

~ 

42,071.52 

2000 

0 

— 

— 

— 

— 

Six- Year  Total 

102,323.80 

Operations  Management 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

0 

— 

— 

~ 

~ 

1996 

8 

$  23,244.49 

$  5,447.50 

$  14,400.50 

$  115,203.99 

1997 

10 

35,984.39 

554.19 

14,939.30 

149,393.02 

1998 

8 

23,301.68 

6,773.28 

12,949.42 

103,595.32 

1999 

8 

24,090.28 

640.00 

11,692.44 

93,539.50 

2000 

4 

16,753.46 

5,847.66 

10,907.15 

43,628.61 

Six-Year  Total 

505,360.44 
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Program  Quality  Assurance 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

~ 

— 

— 

$  11,209.10 

1996 

1 

— 

_ 

— 

20,592.90 

1997 

1 

— 

— 

— 

25,872.30 

1998 

0 

— 

— 

— 

— 

1999 

2 

$  15,058.36 

$  1,275.00 

S  8,166.68 

16,333.36 

2000 

2 

12,325.44 

1,050.00 

6,687.72 

13,375.44 

Six-Year  Total 

87,383.10 

Reading 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

14 

$  50,237.25 

$    2,999.10 

$  34,505.28 

$  483,073.85 

1996 

9 

57,072.48 

23,054.50 

38,544.90 

346,904.08 

1997 

4 

47,702.88 

14,987.88 

32,022.16 

128,088.63 

1998 

1 

— 

— 

— 

4,058.02 

1999 

1 

— 

— 

— 

4,449.63 

2000 

0 

— 

— 

— 

— 

Six- Year  Total 

966,574.21 

School  Business 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

— 

— 

$        8,025.00 

1996 

2 

$  29,136.20 

S  11,074.50 

$  20,105.35 

40,210.70 

1997 

2 

40,549.00 

9,876.60 

25,212.80 

50,425.60 

1998 

2 

43,488.32 

14,117.50 

28,802.91 

57,605.82 

1999 

2 

46,361.07 

14,989.44 

30,675.26 

61,350.51 

2000 

1 

— 

— 

— 

25,921.66 

Six- Year  Total 

243,539.29 

School  to  Career 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

— 

— 

— 

S  32,595.42 

1996 

2 

$  8,988.00 

$  7,848.16 

S  8,418.08 

16,836.16 

1997 

0 

— 

~ 

— 

— 

1998 

0 

— 

— 

— 

— 

1999 

0 

— 

— 

— 

— 

2000 

0 

— 

— 

— 

— 

Six- Year  Total 

49,431.58 
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Special  Education  Appeals 


Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

3 

$  37,557.00 

S  5,544.00 

$  16,457.07 

$  49,371.20 

1996 

5 

32,340.75 

8,437.50 

15,984.45 

79,922.25 

1997 

5 

44,035.36 

14,241.81 

30,486.19 

152,430.95 

1998 

3 

39,711.00 

37,691.25 

38,767.52 

116,302.57 

1999 

0 

~ 

— 

~ 

~ 

2000 

0 

~ 

— 

— 

~ 

Six- Year  Total 

398,026.97 

Special  Services 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

1 

~ 

— 

~ 

$  30,494.98 

1996 

0 

~ 

— 

~ 

— 

1997 

0 

— 

— 

~ 

~ 

1998 

0 

— 

~ 

~ 

~ 

1999 

0 

— 

— 

— 

~ 

2000 

0 

— 

~ 

~ 

~ 

Six- Year  Total 

30,494.98 

Student  Assessment 

Fiscal  Year 

Number  of 
Consultants 

Maximum 
Payment 

Minimum 
Payment 

Average 
Payment 

Total 

Consultant 

Payments 

1995 

6 

$     40,221.30 

$     3,141.00 

$       19,959.13 

$      119,754.80 

1996 

7 

40,060.80 

3,632.25 

18,230.54 

127,613.75 

1997 

4 

21,840.12 

17,365.79 

20,467.13 

81,868.52 

1998 

2 

46,402.50 

43,148.00 

44,775.25 

89,550.50 

1999 

2 

51,426.10 

1,463.40 

26,444.75 

52,889.50 

2000 

2 

36,600.00 

26,890.96 

31,745.48 

63,490.96 

Six-Year  Total 

535,168.03 
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